Representing Embedded Architectures at the Kernel Summit

Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

Hi All,

We've got to the point where there are simply too many embedded
architectures to invite all the arch maintainers to the kernel summit.
So, this year, we thought we'd do embedded via topic driven invitations
instead.  So what we're looking for is a proposal to discuss the issues
most affecting embedded architectures, or preview any features affecting
the main kernel which embedded architectures might need ... or any other
topics from embedded architectures which might need discussion or
debate.  If you'd like to do this, could you either reply to this email,
or send proposals to:

ksummit-2009-discuss@lists.linux-foundation.org

James

Re: Representing Embedded Architectures at the Kernel Summit

[Josh Boyer]

On Tue, Jun 02, 2009 at 03:22:20PM +0000, James Bottomley wrote:
>Hi All,
>
>We've got to the point where there are simply too many embedded
>architectures to invite all the arch maintainers to the kernel summit.
>So, this year, we thought we'd do embedded via topic driven invitations
>instead.  So what we're looking for is a proposal to discuss the issues
>most affecting embedded architectures, or preview any features affecting
>the main kernel which embedded architectures might need ... or any other
>topics from embedded architectures which might need discussion or
>debate.  If you'd like to do this, could you either reply to this email,
>or send proposals to:
>
>ksummit-2009-discuss@lists.linux-foundation.org

For those not following the ksummit list, below is the current list of
suggested topics:

PROPOSER        TOPIC

Jon Corbet      How much do we owe sloppy application writers
Jon Corbet      The containers end game and how we get there
Balbir Singh    The Hacking Hour
Rafael Wysocki  Regression tracking and kernel quality
Jon Corbet      Criteria for acceptance of kernel tracepoints
Jesse Barnes    Profiling and performance counters
Eric Biederman  Procedures for dealing with patent problems
John Linville   Patch review checklist
Matthew Wilcox  How to handle style-only patches
Dan Williams    RAID unification / stacked block devices
Jiri Kosina     User-space drivers - worth encouraging?
Sam Ravnborg    Shipping user-space components in the kernel
Kay Sievers     Establishment/maintenance of per-subsystem todo lists
Steve Rostedt   Improving changelogs
Jon Corbet      I/O bandwidth controllers (maybe minisummit report)
Ted Ts'o        PulseAudio and kernel interface issues
Greg KH         Generic architecture support and arch layer cleanup
                (Josh Boyer: add device tree discussion?)
Dave Jones      cpumasks, churn, and unending API changes
Dirk Hohndel    Issues related to wireless technologies
Jon Corbet      Tracing:
                        Merging utrace
                        Ftrace mainline v. private debugging tools
                        Non-ftrace visibility infrastructure
                        Systemtap for kernel developers
Marcel Holtmann         Tracing and protocol dumping



Some of these certainly impact embedded architectures and workloads.  Examples
would be "Generic architecture support, arch layer cleanup", tracing, profiling
and performance counters, userspace drivers, etc.

Which leads me to suggest that it is at least worth having someone with an
embedded focus at KS to simply keep an eye out for impacts of generic changes.
"Feature parity" is something I often deal with in trying to keep ppc4xx up to
speed with the rest of the archs in the kernel.

josh

Re: Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Tue, 2009-06-02 at 13:29 -0400, Josh Boyer wrote:
> Which leads me to suggest that it is at least worth having someone with an
> embedded focus at KS to simply keep an eye out for impacts of generic changes.
> "Feature parity" is something I often deal with in trying to keep ppc4xx up to
> speed with the rest of the archs in the kernel.

We're fine with this, if that's how the embedded guys would like to do
it ... how do you want to nominate the "someone with an embedded focus"?
We chose the topic driven approach because that's the one it's easiest
for the Kernel Summit Programme Committee to look at and make attendance
decisions based upon.  However, if all the embedded people want to
choose their own representatives, that's fine by us too ... as long as
you can devise a fair process.

James

Re: Representing Embedded Architectures at the Kernel Summit

[David VomLehn]

On Tue, Jun 02, 2009 at 12:42:57PM -0500, James Bottomley wrote:
> On Tue, 2009-06-02 at 13:29 -0400, Josh Boyer wrote:
> > Which leads me to suggest that it is at least worth having someone with an
> > embedded focus at KS to simply keep an eye out for impacts of generic changes.
> > "Feature parity" is something I often deal with in trying to keep ppc4xx up to
> > speed with the rest of the archs in the kernel.
> 
> We're fine with this, if that's how the embedded guys would like to do
> it ... how do you want to nominate the "someone with an embedded focus"?
> We chose the topic driven approach because that's the one it's easiest
> for the Kernel Summit Programme Committee to look at and make attendance
> decisions based upon.  However, if all the embedded people want to
> choose their own representatives, that's fine by us too ... as long as
> you can devise a fair process.

Now, James, I think you haven't been paying attention to the embedded Linux
world...asking us to devise *any* process is much more chaotic than herding
cats. Expecting something everyone agrees is fair would probably take until
at least KS 2010. That being said, we have three people who are listed in
MAINTAINERS under embedded Linux. We might start by seeing which of them
might take up the mantle and vote on the linux-embedded mailing list.

> James

Re: Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Tue, 2009-06-02 at 10:52 -0700, David VomLehn wrote:
> On Tue, Jun 02, 2009 at 12:42:57PM -0500, James Bottomley wrote:
> > On Tue, 2009-06-02 at 13:29 -0400, Josh Boyer wrote:
> > > Which leads me to suggest that it is at least worth having someone with an
> > > embedded focus at KS to simply keep an eye out for impacts of generic changes.
> > > "Feature parity" is something I often deal with in trying to keep ppc4xx up to
> > > speed with the rest of the archs in the kernel.
> > 
> > We're fine with this, if that's how the embedded guys would like to do
> > it ... how do you want to nominate the "someone with an embedded focus"?
> > We chose the topic driven approach because that's the one it's easiest
> > for the Kernel Summit Programme Committee to look at and make attendance
> > decisions based upon.  However, if all the embedded people want to
> > choose their own representatives, that's fine by us too ... as long as
> > you can devise a fair process.
> 
> Now, James, I think you haven't been paying attention to the embedded Linux
> world...asking us to devise *any* process is much more chaotic than herding
> cats. Expecting something everyone agrees is fair would probably take until
> at least KS 2010. That being said, we have three people who are listed in
> MAINTAINERS under embedded Linux. We might start by seeing which of them
> might take up the mantle and vote on the linux-embedded mailing list.

Even for someone as inattentive as me, the general problems of getting
embedded people to agree the sky is blue did impinge on the peripheral
consciousness.  Thus: If you can come up with such a process in a timely
fashion then fine ... if not, we'll do the topic based one suggested by
the PC.

James

Re: Representing Embedded Architectures at the Kernel Summit

[Josh Boyer]

On Tue, Jun 02, 2009 at 01:25:32PM -0500, James Bottomley wrote:
>On Tue, 2009-06-02 at 10:52 -0700, David VomLehn wrote:
>> On Tue, Jun 02, 2009 at 12:42:57PM -0500, James Bottomley wrote:
>> > On Tue, 2009-06-02 at 13:29 -0400, Josh Boyer wrote:
>> > > Which leads me to suggest that it is at least worth having someone with an
>> > > embedded focus at KS to simply keep an eye out for impacts of generic changes.
>> > > "Feature parity" is something I often deal with in trying to keep ppc4xx up to
>> > > speed with the rest of the archs in the kernel.
>> > 
>> > We're fine with this, if that's how the embedded guys would like to do
>> > it ... how do you want to nominate the "someone with an embedded focus"?
>> > We chose the topic driven approach because that's the one it's easiest
>> > for the Kernel Summit Programme Committee to look at and make attendance
>> > decisions based upon.  However, if all the embedded people want to
>> > choose their own representatives, that's fine by us too ... as long as
>> > you can devise a fair process.
>> 
>> Now, James, I think you haven't been paying attention to the embedded Linux
>> world...asking us to devise *any* process is much more chaotic than herding
>> cats. Expecting something everyone agrees is fair would probably take until
>> at least KS 2010. That being said, we have three people who are listed in
>> MAINTAINERS under embedded Linux. We might start by seeing which of them
>> might take up the mantle and vote on the linux-embedded mailing list.
>
>Even for someone as inattentive as me, the general problems of getting
>embedded people to agree the sky is blue did impinge on the peripheral
>consciousness.  Thus: If you can come up with such a process in a timely
>fashion then fine ... if not, we'll do the topic based one suggested by
>the PC.

Most of these are probably tired and old, but some possible topic suggestions:

1) Kernel binary bloat (again)

2) Encouraging upstream participation of "Embedded" distros

Things like Moblin and Android are getting a lot of press these days, but
embedded distros have been around for a while.  Are we getting good
participation from these vendors?  Is there something we could be doing to
encourage such participation?  Has CELF helped with this at all?  etc

3) Netbooks - the bridge between embedded and desktop?

Is the flourish of low cost netbooks, some pre-installed with Linux, having any
impact on how we review and develop general kernel code?

One of the problems I struggle with is coming up with embedded topics general
enough to be of interest to a broader set of both upstream kernel developers
and the embedded community.  Those that are general enough have either been
discussed quite a bit already, or are already on the list of topics.

Maybe I'm just not trying hard enough.

josh

Re: Representing Embedded Architectures at the Kernel Summit

[Tim Bird]

Josh Boyer wrote:
> 2) Encouraging upstream participation of "Embedded" distros
> 
> Things like Moblin and Android are getting a lot of press these days, but
> embedded distros have been around for a while.  Are we getting good
> participation from these vendors?  Is there something we could be doing to
> encourage such participation?  Has CELF helped with this at all?  etc

CELF tries, but the progress is exceedingly slow.  Recently
we've been more focused on contracting specific feature work.
(E.g. Squashfs mainlining).

James Bottomley wrote:
> Even for someone as inattentive as me, the general problems of getting
> embedded people to agree the sky is blue did impinge on the peripheral
> consciousness.  Thus: If you can come up with such a process in a timely
> fashion then fine ... if not, we'll do the topic based one suggested by
> the PC.

With regard to a process to determine representatives, I'm not
sure we need one.  Based on participation and inclusion in
MAINTAINERS, either Matt Mackall or David Woodhouse can
represent most embedded issues just fine.  And I can say that
officially on behalf of CELF and it's members, which would
account for a large fraction of the overall embedded community.

With regard to topics, do topics drive attendee invitations,
or vice-versa?

Here's my own issue list:

tracing - already well (over?) represented

bloat - tracing will help identify performance bloat.
As for size bloat, a smaller kernel is always desirable, but we
are seeing signs that Moore's law is catching up and making
this less an issue (for the kernel - apps still have big
problems here.)

power management - Use cases for products that spend most
of their time off (even while appearing to be running) are
of interest. I don't know what the status 'wakelock-like'
solutions is.

fast boot - kernel is almost done? (!!!)  The new target for
kernel boot time is 300 milliseconds.  Once there, almost
all problems are then user space issues.  It is interesting
how much of a differentiator fast boot became for Linux
in netbooks and dual-boot configurations, in just the last
2 years - which just shows that sometimes it pays off to
optimize something. ;-)

participation - talking about this is like beating a dead horse
(for me at least).  I've been working on this for 5 years now,
making baby steps forward.  The issues are, by now, well understood
(I hope).  I'm not sure what a KS discussion is going to do
to drive issues here.

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Tue, 2009-06-02 at 12:30 -0700, Tim Bird wrote:
> Josh Boyer wrote:
> > 2) Encouraging upstream participation of "Embedded" distros
> > 
> > Things like Moblin and Android are getting a lot of press these days, but
> > embedded distros have been around for a while.  Are we getting good
> > participation from these vendors?  Is there something we could be doing to
> > encourage such participation?  Has CELF helped with this at all?  etc
> 
> CELF tries, but the progress is exceedingly slow.  Recently
> we've been more focused on contracting specific feature work.
> (E.g. Squashfs mainlining).
> 
> James Bottomley wrote:
> > Even for someone as inattentive as me, the general problems of getting
> > embedded people to agree the sky is blue did impinge on the peripheral
> > consciousness.  Thus: If you can come up with such a process in a timely
> > fashion then fine ... if not, we'll do the topic based one suggested by
> > the PC.
> 
> With regard to a process to determine representatives, I'm not
> sure we need one.  Based on participation and inclusion in
> MAINTAINERS, either Matt Mackall or David Woodhouse can
> represent most embedded issues just fine.  And I can say that
> officially on behalf of CELF and it's members, which would
> account for a large fraction of the overall embedded community.
> 
> With regard to topics, do topics drive attendee invitations,
> or vice-versa?
> 
> Here's my own issue list:
> 
> tracing - already well (over?) represented
> 
> bloat - tracing will help identify performance bloat.
> As for size bloat, a smaller kernel is always desirable, but we
> are seeing signs that Moore's law is catching up and making
> this less an issue (for the kernel - apps still have big
> problems here.)
> 
> power management - Use cases for products that spend most
> of their time off (even while appearing to be running) are
> of interest. I don't know what the status 'wakelock-like'
> solutions is.
> 
> fast boot - kernel is almost done? (!!!)  The new target for
> kernel boot time is 300 milliseconds.  Once there, almost
> all problems are then user space issues.  It is interesting
> how much of a differentiator fast boot became for Linux
> in netbooks and dual-boot configurations, in just the last
> 2 years - which just shows that sometimes it pays off to
> optimize something. ;-)

OK, if that's what you all want, that's what we can do ... however, it
would likely be the same people discussing the same issues.

> participation - talking about this is like beating a dead horse
> (for me at least).  I've been working on this for 5 years now,
> making baby steps forward.  The issues are, by now, well understood
> (I hope).  I'm not sure what a KS discussion is going to do
> to drive issues here.

This is what made us suggest the presentation driven approach.  We can
send people who understand how the kernel development process out
anointed as embedded maintainers.  However, looking at the arch
directory, you have a ton of new kids on the block.  We wondered if,
perhaps, rather than having seasoned kernel developers reach out to the
embedded community, we might try giving the embedded community the
opportunity to reach out to us.  The topic of "flattened device tree"
look interesting to me (perhaps because I'm a hardened device driver
person and things like that always look interesting to me) ... if we can
get a few more like that out of the woodwork, this approach might end up
being successful.

James

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Bill Gatliff]

James Bottomley wrote:
> On Tue, 2009-06-02 at 12:30 -0700, Tim Bird wrote:
>   
>> With regard to a process to determine representatives, I'm not
>> sure we need one.  Based on participation and inclusion in
>> MAINTAINERS, either Matt Mackall or David Woodhouse can
>> represent most embedded issues just fine.  And I can say that
>> officially on behalf of CELF and it's members, which would
>> account for a large fraction of the overall embedded community.
>>     

What about others like myself who would like to get more involved, but 
don't show up in the MAINTAINERS list?  I might be interested in lending 
a hand in helping to represent the embedded issues...


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Robert Schwebel]

On Tue, Jun 02, 2009 at 03:37:44PM -0500, James Bottomley wrote:
> The topic of "flattened device tree" look interesting to me (perhaps
> because I'm a hardened device driver person and things like that
> always look interesting to me) ...

The recent oftree activities look indeed very promising; the different
boot-information-passing methods, mainly in powerpc and arm land, is
IMHO and important field where a generic kernel infrastructure would
make sense for the embedded people. Taken that oftree has created
robustness and compatiblity problems in the past but seems to move into
a good direction recently, feedback from core kernel developers would
certainly be a good thing.

> if we can get a few more like that out of the woodwork, this approach
> might end up being successful.

Could flickerfree-bootsplash be a topic? Or is that completely pushed
into the userspace these fastboot days?

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Greg KH]

On Tue, Jun 02, 2009 at 11:34:52PM +0200, Robert Schwebel wrote:
> Could flickerfree-bootsplash be a topic? Or is that completely pushed
> into the userspace these fastboot days?

We have that working today, no in-kernel work needed other than the
already-present KMS stuff.  See the recent Moblin images for proof of
this.

thanks,

greg k-h

Re: flicker free booting

[Robert Schwebel]

On Tue, Jun 02, 2009 at 08:35:35PM -0700, Greg KH wrote:
> On Tue, Jun 02, 2009 at 11:34:52PM +0200, Robert Schwebel wrote:
> > Could flickerfree-bootsplash be a topic? Or is that completely
> > pushed into the userspace these fastboot days?
>
> We have that working today, no in-kernel work needed other than the
> already-present KMS stuff. See the recent Moblin images for proof of
> this.

Well, I assume that the issue you've addressed is being flicker free
between the kernel and x.org; what I mean is flicker-free even with the
bootloader in mind.

What we want to address is another problem which is present on almost
all display-enabled SoC-type embedded hardware. This kind of hardware is
BIOS-free and is booted from an embedded bootloader like u-boot. These
processors are often pretty slow, being in the 200 MHz ARM926 league.

The boot scenario on these boxes looks like this:

1)  power-on
2a) on NAND-only systems, pre-bootloader is fetched from NAND block
2b) bootloader starts from NAND or NOR flash
3)  the bootloader fetches Linux from NAND/NOR flash
4)  kernel starts
5)  userspace starts
6)  application starts (i.e. app against qt+dfb or gtk+dfb)

These SoCs are usually built into "devices" which don't look like PCs;
think of it as mobile phones (although our applications are usually more
industry-style, like machine controllers, measurement systems etc). Most
industry applications have been built with microcontrollers + 2x16 alpha
numeric displays in the last generation; this type of hardware has a
boot time (from power-on to full operation) of something like 0.5 s.

So what I would like to see is this:

a) power on
b) almost immediately (< 1s) a splash screen appears
c) optionally be able to do some animation, progress bar etc
d) application appears (instantanously or by fading)

At the moment, we do this by using random hacks in the kernel, which is
pretty bad. So what I'm searching for is a clean API which may be
accesptable form the Linux mainline.

Here's a proposal from my kernel team:

- u-boot initializes framebuffer on some <address> + shows image
- kernel commandline: mem=127MB framebuffer=<num>:<address>:<size>
  The framebuffer is reserved at the given address (i.e. 1 MB
  at the upper end of the memory, so we have to reduce the
  overall memory available for Linux)
- kernel driver recognizes that the framebuffer was already set up and
  skips the initialisation step (which would otherwhise result in
  flicker)

A second step could be:

- Have this initial framebuffer as /dev/fb0 (mem at the end of RAM)
- In order to avoid flicker, the driver sets up a 2nd framebuffer,
  in dynamically allocated memory -> /dev/fb1. This framebuffer
  continues to display the splash image (or an animation). Switching
  from fb0 -> fb1 is done during VSync.
- While the splash image is still visible in fb1, the application
  starts, up to the point where it shows it's main screen. This usually
  never works flicker-free, but we don't see it, as fb1 is visible.
- switch back to fb0 (during a VSync to avoid flicker)
- optionally discard fb1 ressources

On some hardware, the switch could be done by fading from fb1 to fb0
(i.e. on i.MX27 this should be possible).

Drivers which see the framebuffer= parameter would have to skip their
hardware initialisation step.

What do you think? Would something like this be acceptable? Other ideas?

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: flicker free booting

[David VomLehn]

On Fri, Jul 31, 2009 at 05:53:52PM +0200, Robert Schwebel wrote:
> On Tue, Jun 02, 2009 at 08:35:35PM -0700, Greg KH wrote:
> > On Tue, Jun 02, 2009 at 11:34:52PM +0200, Robert Schwebel wrote:
> > > Could flickerfree-bootsplash be a topic? Or is that completely
> > > pushed into the userspace these fastboot days?
...
> So what I would like to see is this:
> 
> a) power on
> b) almost immediately (< 1s) a splash screen appears
> c) optionally be able to do some animation, progress bar etc
> d) application appears (instantanously or by fading)

This is an issue that I think is common to many embedded platforms, but I'm
not sure this is the solution. (I'm not sure it's *not* the solution, either).
The question I've been looking at is: do sub-second boot times make all this
a moot point? After all, if you can bring up your graphics driver in a fraction
of a second and use *it* to display a splash screen, it seems like it would
meet your need to give almost immediate feedback to users that the system is
alive.
--
David VomLehn

Re: flicker free booting

[Robert Schwebel]

Hi David,

On Fri, Jul 31, 2009 at 11:03:10AM -0700, David VomLehn wrote:
> On Fri, Jul 31, 2009 at 05:53:52PM +0200, Robert Schwebel wrote:
> > On Tue, Jun 02, 2009 at 08:35:35PM -0700, Greg KH wrote:
> > > On Tue, Jun 02, 2009 at 11:34:52PM +0200, Robert Schwebel wrote:
> > > > Could flickerfree-bootsplash be a topic? Or is that completely
> > > > pushed into the userspace these fastboot days?
> ...
> > So what I would like to see is this:
> > 
> > a) power on
> > b) almost immediately (< 1s) a splash screen appears
> > c) optionally be able to do some animation, progress bar etc
> > d) application appears (instantanously or by fading)
> 
> This is an issue that I think is common to many embedded platforms, but I'm
> not sure this is the solution. (I'm not sure it's *not* the solution, either).

I hope it is more a call-for-discussion :-)

> The question I've been looking at is: do sub-second boot times make
> all this a moot point? After all, if you can bring up your graphics
> driver in a fraction of a second and use *it* to display a splash
> screen, it seems like it would meet your need to give almost immediate
> feedback to users that the system is alive.

Yup, that would be cool.

However, on flash-based systems in the ARM926 / 200...400 MHz class
(which is still quite common) the boot time up to the point where the
penguin appears is still about 3...4 s [1] which is too long. So I
suspect until systems become faster, we'll have no other choice than
such as scenario.

rsc

[1] we can do some benchmarking next week
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: flicker free booting

[Bill Gatliff]

Robert Schwebel wrote:
>   
>> The question I've been looking at is: do sub-second boot times make
>> all this a moot point? After all, if you can bring up your graphics
>> driver in a fraction of a second and use *it* to display a splash
>> screen, it seems like it would meet your need to give almost immediate
>> feedback to users that the system is alive.
>>     
>
> Yup, that would be cool.
>
> However, on flash-based systems in the ARM926 / 200...400 MHz class
> (which is still quite common) the boot time up to the point where the
> penguin appears is still about 3...4 s [1] which is too long. So I
> suspect until systems become faster, we'll have no other choice than
> such as scenario.
>   

Could your bootloader pre-initialize the graphics hardware to the same 
mode that the Linux driver will ultimately select, and then throw up a 
static graphic?  That would give you some output until the driver itself 
comes online.

And if the driver was in a demand-loaded module, then the static image 
wouldn't disappear until just before the application that does the 
animation was ready to start drawing.



b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: flicker free booting

[Sascha Hauer]

On Fri, Jul 31, 2009 at 01:42:35PM -0500, Bill Gatliff wrote:
> Robert Schwebel wrote:
>>   
>>> The question I've been looking at is: do sub-second boot times make
>>> all this a moot point? After all, if you can bring up your graphics
>>> driver in a fraction of a second and use *it* to display a splash
>>> screen, it seems like it would meet your need to give almost immediate
>>> feedback to users that the system is alive.
>>>     
>>
>> Yup, that would be cool.
>>
>> However, on flash-based systems in the ARM926 / 200...400 MHz class
>> (which is still quite common) the boot time up to the point where the
>> penguin appears is still about 3...4 s [1] which is too long. So I
>> suspect until systems become faster, we'll have no other choice than
>> such as scenario.
>>   
>
> Could your bootloader pre-initialize the graphics hardware to the same  
> mode that the Linux driver will ultimately select, and then throw up a  
> static graphic?  That would give you some output until the driver itself  
> comes online.

I implemented exactly this for the i.MX framebuffer last week. It works
by looking at the controllers physical screen start address, ioremapping
this address and copying the content to the newly allocated framebuffer.
Currently the driver still reinitilializes the controller with exactly
the same values. We could skip this initialization step depending on a
imxfb.preinitialized=1 command line parameter and read back
resolution/depth from the device in order to fill the fb_info struct.

This approach works good, I just wonder how acceptable it is for
mainline.

The downside of this approach is that we have to add
mem=${physical_mem - 1MB} to the command line.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: flicker free booting

[Geert Uytterhoeven]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset="ks_c_5601-1987", Size: 1714 bytes --]

On Mon, 3 Aug 2009, Sascha Hauer wrote:
> On Fri, Jul 31, 2009 at 01:42:35PM -0500, Bill Gatliff wrote:
> > Could your bootloader pre-initialize the graphics hardware to the same  
> > mode that the Linux driver will ultimately select, and then throw up a  
> > static graphic?  That would give you some output until the driver itself  
> > comes online.
> 
> I implemented exactly this for the i.MX framebuffer last week. It works
> by looking at the controllers physical screen start address, ioremapping
> this address and copying the content to the newly allocated framebuffer.
> Currently the driver still reinitilializes the controller with exactly
> the same values. We could skip this initialization step depending on a
> imxfb.preinitialized=1 command line parameter and read back
> resolution/depth from the device in order to fill the fb_info struct.
> 
> This approach works good, I just wonder how acceptable it is for
> mainline.

If the actual frame buffer driver doesn't do much after initialization, we
already have (at least) two similar drivers in the kernel: vesafb and offb.
Both rely on firmware (PC BIOS vs. Open Firmware) to initialize the graphics
mode. Feel free to unify all of this into `staticfb.c'...

With kind regards,

Geert Uytterhoeven
Software Architect
Techsoft Centre

Technology and Software Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

Re: flicker free booting

[Bill Gatliff]

David VomLehn wrote:
> This is an issue that I think is common to many embedded platforms, but I'm
> not sure this is the solution. (I'm not sure it's *not* the solution, either).
> The question I've been looking at is: do sub-second boot times make all this
> a moot point? After all, if you can bring up your graphics driver in a fraction
> of a second and use *it* to display a splash screen, it seems like it would
> meet your need to give almost immediate feedback to users that the system is
> alive.
>   

Those fractions-of-seconds boot times are beyond the reach of the 200 
MHz-class ARM9 processors and similar, where it takes two or three 
seconds just to load and uncompress the kernel from NOR or NAND flash.

And add a second or so on top of that to zeroinit the graphics 
framebuffer, if your display is of any decent size and depth.


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: flicker free booting

[Tim Bird]

Bill Gatliff wrote:
> 
> Those fractions-of-seconds boot times are beyond the reach of the 200
> MHz-class ARM9 processors and similar, where it takes two or three
> seconds just to load and uncompress the kernel from NOR or NAND flash.
While I don't disagree from a practical standpoint - at Sony
using XIP we have been able to finish kernel boot on a 192 MHZ ARM9
in 186 milliseconds.  In the lab, anyway?

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: flicker free booting

[Bill Gatliff]

Tim Bird wrote:
> Bill Gatliff wrote:
>   
>> Those fractions-of-seconds boot times are beyond the reach of the 200
>> MHz-class ARM9 processors and similar, where it takes two or three
>> seconds just to load and uncompress the kernel from NOR or NAND flash.
>>     
> While I don't disagree from a practical standpoint - at Sony
> using XIP we have been able to finish kernel boot on a 192 MHZ ARM9
> in 186 milliseconds.  In the lab, anyway?
>   

Yea, that's an upside of XIP: no uncompression stage.  :)

I wonder, though, if the bandwidth limitations of NOR flash make the 
system slower overall?  I guess the bootloader could copy an 
uncompressed image from flash into RAM, and then XIP it from there to 
get the best of both worlds, if you could stand the time hit of the 
memcpy...


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: flicker free booting

[Robert Schwebel]

On Fri, Jul 31, 2009 at 12:48:37PM -0700, Tim Bird wrote:
> > Those fractions-of-seconds boot times are beyond the reach of the
> > 200 MHz-class ARM9 processors and similar, where it takes two or
> > three seconds just to load and uncompress the kernel from NOR or
> > NAND flash.
>
> While I don't disagree from a practical standpoint - at Sony using XIP
> we have been able to finish kernel boot on a 192 MHZ ARM9 in 186
> milliseconds. In the lab, anyway?

Wow, that's pretty fast; if that would be possible for standard
scenarios, it would indeed be better to do everything on the kernel
side.

Can you elaborate about the details of that experiment? Was that normal
XIP from NOR? What size has this kernel been?

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: flicker free booting

[Tim Bird]

Robert Schwebel wrote:
> On Fri, Jul 31, 2009 at 12:48:37PM -0700, Tim Bird wrote:
>>> Those fractions-of-seconds boot times are beyond the reach of the
>>> 200 MHz-class ARM9 processors and similar, where it takes two or
>>> three seconds just to load and uncompress the kernel from NOR or
>>> NAND flash.
>> While I don't disagree from a practical standpoint - at Sony using XIP
>> we have been able to finish kernel boot on a 192 MHZ ARM9 in 186
>> milliseconds. In the lab, anyway?
> 
> Wow, that's pretty fast; if that would be possible for standard
> scenarios, it would indeed be better to do everything on the kernel
> side.
> 
> Can you elaborate about the details of that experiment? Was that normal
> XIP from NOR? What size has this kernel been?

This was done quite some time ago, and it looks like my memory was
not too good.  According to slides I did at the time, it was
actually 110 milliseconds.  I'm sure this was a 2.4 kernel (likely
2.4.20).  I couldn't find the size of the kernel used in the test, but
I think it was about 800k, uncompressed.  (I'm not positive, though.)
It was normal XIP from NOR flash.

See slides 21 and 22 in the following:
http://elinux.org/images/7/78/ReducingStartupTime_v0.8.pdf

Here is some salient data:

                  With Compression   W/ O Compression      XIP
Copy              56 msec          120 msec             0 msec
Decompression    545 msec            0 msec             0 msec
Kernel execution  88 msec           88 msec           110 msec
Total:           689 msec          208 msec           110 msec

Note that copy time went up when going from a compressed to an
uncompressed kernel.  Decompression time went down (to 0, no
surpise).  When XIP was used, both copy time and decompression
time were eliminated, but kernel execution time for the boot
went from 88 msec to 110 msec.  So XIP did incur a pretty
hefty runtime penalty.

Aaahhh, 2.4.  The kernel was simpler in those days... :-)

 -- Tim


=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: flicker free booting

[Bill Gatliff]

Robert Schwebel wrote:
> A second step could be:
>
> - Have this initial framebuffer as /dev/fb0 (mem at the end of RAM)
> - In order to avoid flicker, the driver sets up a 2nd framebuffer,
>   in dynamically allocated memory -> /dev/fb1. This framebuffer
>   continues to display the splash image (or an animation). Switching
>   from fb0 -> fb1 is done during VSync.
>   

What about double-buffering fb0 instead?  That's how Android works.

> Drivers which see the framebuffer= parameter would have to skip their
> hardware initialisation step.
>   

Actually, I'd rather that drivers look at the hardware itself, and 
verify that the configuration matches what the parameter specifies 
before making changes.  That way you could use framebuffer= to 
communicate the desired setup to the driver in cases where the hardware 
wasn't already initialized.



-- 
Bill Gatliff
bgat@billgatliff.com

Re: flicker free booting

[Jamie Lokier]

Bill Gatliff wrote:
> Actually, I'd rather that drivers look at the hardware itself, and 
> verify that the configuration matches what the parameter specifies 
> before making changes.  That way you could use framebuffer= to 
> communicate the desired setup to the driver in cases where the hardware 
> wasn't already initialized.

It's a good idea when possible, but some hardware cannot be queried in
enough detail to confirm the framebuffer configuration - unless the
bootloader passes an extra parameter saying what it has done.

-- Jamie

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[David VomLehn]

On Tue, Jun 02, 2009 at 03:37:44PM -0500, James Bottomley wrote:
...
> This is what made us suggest the presentation driven approach.  We can
> send people who understand how the kernel development process out
> anointed as embedded maintainers.  However, looking at the arch
> directory, you have a ton of new kids on the block.  We wondered if,
> perhaps, rather than having seasoned kernel developers reach out to the
> embedded community, we might try giving the embedded community the
> opportunity to reach out to us.  The topic of "flattened device tree"
> look interesting to me (perhaps because I'm a hardened device driver
> person and things like that always look interesting to me) ... if we can
> get a few more like that out of the woodwork, this approach might end up
> being successful.

Failure reporting is the one area where embedded applications have
little overlap with other Linux application domains. The cable settop box
environment has:
o Limited peristent storage
o Low or no upstream bandwidth
o Little access to hundreds of thousands of devices in the field

When a kernel panics in the field, we have no place to put a core dump
and, if we had a place to put it, it would take way too long to upload
it when the box comes back up. And most people just don't understand when
you knock at their door at midnight, JTAG probe in hand.

We hook in a panic notifier and have it generate a really rich report.
At present, this report stays in memory until we reboot and send it
upstream (or write it to flash), but we could really write it to any
device with which we can use polled I/O (interrupts being questionable
at this point). Generic interfaces to support this would be useful.

Many embedded devices have highly integrated stacks, so failures in user
space lead to device reboots, and you want to leverage much of the same
ability to store and send failure reports.

Our failure report includes things you'd expect as well as various pieces
of history, such as:
o IRQs
o softirq dispatches (including max times)
o selected /proc info, e.g. /proc/meminfo

We also report info on the current thread, like backtracing and
/proc/<pid>/maps, though I'm not sure it's as useful as it might be.

Though I'm working on pushing this stuff out, other things that might be
helpful are:
o If you get to panic() by way of die(), you've lost the registers passed to
  die(). We save a pointer off, but it's really a kludge.
o The implementation of die() varies from platform to platform and isn't even
  called die() everywhere.
o It is truly nasty trying to get /proc information when you are in a panic
  situation--any semaphores being held are not going to be released, so you
  have to duplicate a lot of the code, minus the semaphores. Pretty gross
  and there is no way our implementation will be acceptable.
o Increased reporting on what's happening in user/kernel space interaction.
  For example, a signal sent in good faith might kill a buggy process. It
  would be helpful to log signals that result in a process' death.
o Then there is more speculative stuff. For example, your caches would
  have a copy of the most recently accessed code and data.  If your
  processor supports dumping cache, it might help determing what went wrong.

> James

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Eric W. Biederman]

David VomLehn <dvomlehn@cisco.com> writes:

> On Tue, Jun 02, 2009 at 03:37:44PM -0500, James Bottomley wrote:
> ...
>> This is what made us suggest the presentation driven approach.  We can
>> send people who understand how the kernel development process out
>> anointed as embedded maintainers.  However, looking at the arch
>> directory, you have a ton of new kids on the block.  We wondered if,
>> perhaps, rather than having seasoned kernel developers reach out to the
>> embedded community, we might try giving the embedded community the
>> opportunity to reach out to us.  The topic of "flattened device tree"
>> look interesting to me (perhaps because I'm a hardened device driver
>> person and things like that always look interesting to me) ... if we can
>> get a few more like that out of the woodwork, this approach might end up
>> being successful.
>
> Failure reporting is the one area where embedded applications have
> little overlap with other Linux application domains. The cable settop box
> environment has:
> o Limited peristent storage
> o Low or no upstream bandwidth
> o Little access to hundreds of thousands of devices in the field
>
> When a kernel panics in the field, we have no place to put a core dump
> and, if we had a place to put it, it would take way too long to upload
> it when the box comes back up. And most people just don't understand when
> you knock at their door at midnight, JTAG probe in hand.
>
> We hook in a panic notifier and have it generate a really rich report.
> At present, this report stays in memory until we reboot and send it
> upstream (or write it to flash), but we could really write it to any
> device with which we can use polled I/O (interrupts being questionable
> at this point). Generic interfaces to support this would be useful.
>
> Many embedded devices have highly integrated stacks, so failures in user
> space lead to device reboots, and you want to leverage much of the same
> ability to store and send failure reports.
>
> Our failure report includes things you'd expect as well as various pieces
> of history, such as:
> o IRQs
> o softirq dispatches (including max times)
> o selected /proc info, e.g. /proc/meminfo
>
> We also report info on the current thread, like backtracing and
> /proc/<pid>/maps, though I'm not sure it's as useful as it might be.
>
> Though I'm working on pushing this stuff out, other things that might be
> helpful are:
> o If you get to panic() by way of die(), you've lost the registers passed to
>   die(). We save a pointer off, but it's really a kludge.
> o The implementation of die() varies from platform to platform and isn't even
>   called die() everywhere.
> o It is truly nasty trying to get /proc information when you are in a panic
>   situation--any semaphores being held are not going to be released, so you
>   have to duplicate a lot of the code, minus the semaphores. Pretty gross
>   and there is no way our implementation will be acceptable.
> o Increased reporting on what's happening in user/kernel space interaction.
>   For example, a signal sent in good faith might kill a buggy process. It
>   would be helpful to log signals that result in a process' death.
> o Then there is more speculative stuff. For example, your caches would
>   have a copy of the most recently accessed code and data.  If your
>   processor supports dumping cache, it might help determing what went wrong.

Have you looked at doing this with the kexec on panic infrastructure?

Things like mkdumpfile can now have enough information to dump this.

If you are space constrained a stand alone executable could be used
instead of a linux kernel to marshal the information into your buffer.

Eric

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Steven Rostedt]

On Tue, 2 Jun 2009, David VomLehn wrote:
> On Tue, Jun 02, 2009 at 03:37:44PM -0500, James Bottomley wrote:
> Our failure report includes things you'd expect as well as various pieces
> of history, such as:
> o IRQs
> o softirq dispatches (including max times)
> o selected /proc info, e.g. /proc/meminfo
> 
> We also report info on the current thread, like backtracing and
> /proc/<pid>/maps, though I'm not sure it's as useful as it might be.
> 
> Though I'm working on pushing this stuff out, other things that might be
> helpful are:
> o If you get to panic() by way of die(), you've lost the registers passed to
>   die(). We save a pointer off, but it's really a kludge.
> o The implementation of die() varies from platform to platform and isn't even
>   called die() everywhere.
> o It is truly nasty trying to get /proc information when you are in a panic
>   situation--any semaphores being held are not going to be released, so you
>   have to duplicate a lot of the code, minus the semaphores. Pretty gross
>   and there is no way our implementation will be acceptable.
> o Increased reporting on what's happening in user/kernel space interaction.
>   For example, a signal sent in good faith might kill a buggy process. It
>   would be helpful to log signals that result in a process' death.
> o Then there is more speculative stuff. For example, your caches would
>   have a copy of the most recently accessed code and data.  If your
>   processor supports dumping cache, it might help determing what went wrong.

If your system is hooked up to a serial console, another helpful thing to 
do is to pass in "ftrace_dump_on_oops" in the command line and also keep 
the event tracer running, where you enable just the events you are 
interested in. Then if the system crashes, it will dump out the ftrace 
buffer to the console, which could be logged by a serial console and then 
you at least have a trace of the events that lead up to the crash.

The event tracer while active is not that much overhead and could be 
enabled in a production system.

-- Steve

Re: Representing Embedded Architectures at the Kernel Summit

[Jean-Christophe PLAGNIOL-VILLARD]

On 13:29 Tue 02 Jun     , Josh Boyer wrote:
> On Tue, Jun 02, 2009 at 03:22:20PM +0000, James Bottomley wrote:
> >Hi All,
> >
> >We've got to the point where there are simply too many embedded
> >architectures to invite all the arch maintainers to the kernel summit.
> >So, this year, we thought we'd do embedded via topic driven invitations
> >instead.  So what we're looking for is a proposal to discuss the issues
> >most affecting embedded architectures, or preview any features affecting
> >the main kernel which embedded architectures might need ... or any other
> >topics from embedded architectures which might need discussion or
> >debate.  If you'd like to do this, could you either reply to this email,
> >or send proposals to:
> >
> >ksummit-2009-discuss@lists.linux-foundation.org
> 
> For those not following the ksummit list, below is the current list of
> suggested topics:
> 
> PROPOSER        TOPIC
> 
> Jon Corbet      How much do we owe sloppy application writers
> Jon Corbet      The containers end game and how we get there
> Balbir Singh    The Hacking Hour
> Rafael Wysocki  Regression tracking and kernel quality
> Jon Corbet      Criteria for acceptance of kernel tracepoints
> Jesse Barnes    Profiling and performance counters
> Eric Biederman  Procedures for dealing with patent problems
> John Linville   Patch review checklist
> Matthew Wilcox  How to handle style-only patches
> Dan Williams    RAID unification / stacked block devices
> Jiri Kosina     User-space drivers - worth encouraging?
> Sam Ravnborg    Shipping user-space components in the kernel
> Kay Sievers     Establishment/maintenance of per-subsystem todo lists
> Steve Rostedt   Improving changelogs
> Jon Corbet      I/O bandwidth controllers (maybe minisummit report)
> Ted Ts'o        PulseAudio and kernel interface issues
> Greg KH         Generic architecture support and arch layer cleanup
>                 (Josh Boyer: add device tree discussion?)
> Dave Jones      cpumasks, churn, and unending API changes
> Dirk Hohndel    Issues related to wireless technologies
> Jon Corbet      Tracing:
>                         Merging utrace
>                         Ftrace mainline v. private debugging tools
>                         Non-ftrace visibility infrastructure
>                         Systemtap for kernel developers
> Marcel Holtmann         Tracing and protocol dumping
I'd like to propose AMP and kernel relocate
as more and SoC will came with multiple core with or without the same arch

Best Regards,
J.

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Ralf Baechle]

On Wed, Jun 03, 2009 at 12:21:32AM +0200, Jean-Christophe PLAGNIOL-VILLARD wrote:

> I'd like to propose AMP and kernel relocate
> as more and SoC will came with multiple core with or without the same arch

AMP is very interesting.  It's not uncommon two have only a single of like
16 cores running Linux and the others running bare metal code.  A typical
example would be networking code.  Each vendor provides their own AMP
infrastructure and so far it all looks crude.  With an every increasing
number of cores there also is the question if AMP could be generally useful
for the kernel.

  Ralf

Re: Representing Embedded Architectures at the Kernel Summit

[Kumar Gala]

On Jun 2, 2009, at 5:21 PM, Jean-Christophe PLAGNIOL-VILLARD wrote:

> I'd like to propose AMP and kernel relocate
> as more and SoC will came with multiple core with or without the  
> same arch

I think AMP or at least the idea of the kernel communicating with  
other OSes on the same SoC in multi-core systems is interesting.

- k

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Wed, Jun 10, 2009 at 5:13 PM, Kumar Gala<galak@kernel.crashing.org> wrote:
>
> On Jun 2, 2009, at 5:21 PM, Jean-Christophe PLAGNIOL-VILLARD wrote:
>
>> I'd like to propose AMP and kernel relocate
>> as more and SoC will came with multiple core with or without the same arch
>
> I think AMP or at least the idea of the kernel communicating with other OSes
> on the same SoC in multi-core systems is interesting.

Indeed, and not just for SoCs.  I'm currently looking at Ira's
adaptation of virtio to use it for IPC between two separate CPUs
running separate Linux instances and connected via PCI.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Kumar Gala]

On Jun 2, 2009, at 12:29 PM, Josh Boyer wrote:

> Which leads me to suggest that it is at least worth having someone  
> with an
> embedded focus at KS to simply keep an eye out for impacts of  
> generic changes.
> "Feature parity" is something I often deal with in trying to keep  
> ppc4xx up to
> speed with the rest of the archs in the kernel.

I think this is an interesting topic by itself.  How to keep embedded  
(or non-x86) arch's up with features being developed that are core  
kernel.  For example, I know Roland would use 4xx to test his  
infiniband since it was non-coherent system.

- k

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 9:22 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> Hi All,
>
> We've got to the point where there are simply too many embedded
> architectures to invite all the arch maintainers to the kernel summit.
> So, this year, we thought we'd do embedded via topic driven invitations
> instead.  So what we're looking for is a proposal to discuss the issues
> most affecting embedded architectures, or preview any features affecting
> the main kernel which embedded architectures might need ... or any other
> topics from embedded architectures which might need discussion or
> debate.  If you'd like to do this, could you either reply to this email,
> or send proposals to:
>
> ksummit-2009-discuss@lists.linux-foundation.org

Hi James,

One topic that seems to garner debate is the issue of decoupling the
kernel image from the target platform.  ie. On x86, PowerPC and Sparc
a kernel image will boot on any machine (assuming the needed drivers
are enabled), but this is rarely the case in embedded.  Most embedded
kernels require explicit board support selected at compile time with
no way to produce a generic kernel which will boot on a whole family
of devices, let alone the whole architecture.  Part of this is a
firmware issue, where existing firmware passes very little in the way
of hardware description to the kernel, but part is also not making
available any form of common language for describing the machine.

Embedded PowerPC and Microblaze has tackled this problem with the
"Flattened Device Tree" data format which is derived from the
OpenFirmware specifications, and there is some interest and debate (as
discussed recently on the ARM mailing list) about making flattened
device tree usable by ARM also (which I'm currently
proof-of-concepting).  Josh Boyer has already touched on discussing
flattened device tree support at kernel summit in an email to the
ksummit list last week (quoted below), and I'm wondering if a broader
discussing would be warranted.

I think that in the absence of any established standard like the PC
BIOS/EFI or a real Open Firmware interface, then the kernel should at
least offer a recommended interface so that multiplatform kernels are
possible without explicitly having the machine layout described to it
at compile time.  I know that some of the embedded distros are
interested in such a thing since it gets them away from shipping
separate images for each supported board.  ie. It's really hard to do
a generic live-cd without some form of multiplatform.  FDT is a great
approach, but it probably isn't the only option.  It would be worth
debating.

Cheers,
g.

On Thu, May 28, 2009 at 5:41 PM, Josh Boyer <jwboyer@linux.vnet.ibm.com> wrote:
> Not to hijack this entirely, but I'm wondering if we could tie in some of the
> cross-arch device tree discussions that have been taking place among the ppc,
> sparc, and arm developers.
>
> I know the existing OF code for ppc, sparc, and microblaze could probably use
> some consolidation and getting some of the arch maintainers together in a room
> might prove fruitful.  Particularly if we are going to discuss how to make
> drivers work for device tree and standard platforms alike.
>
> josh
>



-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[David VomLehn]

On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
> On Tue, Jun 2, 2009 at 9:22 AM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> > Hi All,
> >
> > We've got to the point where there are simply too many embedded
> > architectures to invite all the arch maintainers to the kernel summit.
> > So, this year, we thought we'd do embedded via topic driven invitations
> > instead.  So what we're looking for is a proposal to discuss the issues
> > most affecting embedded architectures, or preview any features affecting
> > the main kernel which embedded architectures might need ... or any other
> > topics from embedded architectures which might need discussion or
> > debate.  If you'd like to do this, could you either reply to this email,
> > or send proposals to:
> >
> > ksummit-2009-discuss@lists.linux-foundation.org
> 
> Hi James,
> 
> One topic that seems to garner debate is the issue of decoupling the
> kernel image from the target platform.
...
> Embedded PowerPC and Microblaze has tackled this problem with the
> "Flattened Device Tree" data format which is derived from the
> OpenFirmware specifications, and there is some interest and debate (as
> discussed recently on the ARM mailing list) about making flattened
> device tree usable by ARM also (which I'm currently
> proof-of-concepting).  Josh Boyer has already touched on discussing
> flattened device tree support at kernel summit in an email to the
> ksummit list last week (quoted below), and I'm wondering if a broader
> discussing would be warranted.

I absolutely agree with this. We have been using the flattened device tree
on our MIPS platform to support multiple systems, and are close to posting
a patch to the MIPS Linux mailing list. At least one other MIPS platform
has indicated that they want to use the device tree.

> I think that in the absence of any established standard like the PC
> BIOS/EFI or a real Open Firmware interface, then the kernel should at
> least offer a recommended interface so that multiplatform kernels are
> possible without explicitly having the machine layout described to it
> at compile time.  I know that some of the embedded distros are
> interested in such a thing since it gets them away from shipping
> separate images for each supported board.  ie. It's really hard to do
> a generic live-cd without some form of multiplatform.  FDT is a great
> approach, but it probably isn't the only option.  It would be worth
> debating.

Is there another possibility besides the device tree that people are using
on 4 different architectures? We can't mandate support of the device tree
for any particular architecture or platform, but we can make sure the device
tree infrastructure is supported well enough that any architecture can pick
it up easily. Doing so will also encourage support in bootloaders which may
not currently do so.

Should we decide to go this way, there probably a next step wherein we
standardize the device tree entries for those devices that are shared across
multiple architectures and platorms. This will likely be a never-ending
and mostly thankless task, but will again make things easier in the long run.

> Grant Likely, B.Sc., P.Eng.
> Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 11:45 AM, David VomLehn <dvomlehn@cisco.com> wrote:
> Should we decide to go this way, there probably a next step wherein we
> standardize the device tree entries for those devices that are shared across
> multiple architectures and platorms. This will likely be a never-ending
> and mostly thankless task, but will again make things easier in the long run.

devicetree-discuss@ozlabs.org mailing list is your friend here.  In
PowerPC land we've mostly settled on the policy of not merging a
driver before the device tree binding is documented, posted and
reviewed.  This weeds out a lot of common mistakes.

I also think the stuff in Documentation/powerpc/dts-bindings needs to
be moved out to a common location.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Tue, 2009-06-02 at 11:29 -0600, Grant Likely wrote:
> One topic that seems to garner debate is the issue of decoupling the
> kernel image from the target platform.  ie. On x86, PowerPC and Sparc
> a kernel image will boot on any machine (assuming the needed drivers
> are enabled), but this is rarely the case in embedded.  Most embedded
> kernels require explicit board support selected at compile time with
> no way to produce a generic kernel which will boot on a whole family
> of devices, let alone the whole architecture.  Part of this is a
> firmware issue, where existing firmware passes very little in the way
> of hardware description to the kernel, but part is also not making
> available any form of common language for describing the machine.

OK, so my minimal understanding in this area lead me to believe this was
because most embedded systems didn't have properly discoverable busses
and therefore you had to tailor the kernel configuration exactly to the
devices the system had.

> Embedded PowerPC and Microblaze has tackled this problem with the
> "Flattened Device Tree" data format which is derived from the
> OpenFirmware specifications, and there is some interest and debate (as
> discussed recently on the ARM mailing list) about making flattened
> device tree usable by ARM also (which I'm currently
> proof-of-concepting).  Josh Boyer has already touched on discussing
> flattened device tree support at kernel summit in an email to the
> ksummit list last week (quoted below), and I'm wondering if a broader
> discussing would be warranted.
> 
> I think that in the absence of any established standard like the PC
> BIOS/EFI or a real Open Firmware interface, then the kernel should at
> least offer a recommended interface so that multiplatform kernels are
> possible without explicitly having the machine layout described to it
> at compile time.  I know that some of the embedded distros are
> interested in such a thing since it gets them away from shipping
> separate images for each supported board.  ie. It's really hard to do
> a generic live-cd without some form of multiplatform.  FDT is a great
> approach, but it probably isn't the only option.  It would be worth
> debating.

It sounds interesting ... however, it also sounds like an area which
might not impact the core kernel much ... or am I wrong about that?  The
topics we're really looking for the Kernel Summit are ones that require
cross system input and which can't simply be sorted out by organising an
Embedded mini-summit.

Now if flattened device tree could help us out with BIOS, ACPI, EFI and
the other myriad boot and identification standards that seem designed to
hide system information rather than reveal it, then we might be all
ears ...

James

Re: Representing Embedded Architectures at the Kernel Summit

[Mark Brown]

On Tue, Jun 02, 2009 at 05:48:37PM +0000, James Bottomley wrote:
> On Tue, 2009-06-02 at 11:29 -0600, Grant Likely wrote:

> > firmware issue, where existing firmware passes very little in the way
> > of hardware description to the kernel, but part is also not making
> > available any form of common language for describing the machine.

> OK, so my minimal understanding in this area lead me to believe this was
> because most embedded systems didn't have properly discoverable busses
> and therefore you had to tailor the kernel configuration exactly to the
> devices the system had.

Ish; essentially it's pushing the description of the non-enumerable bits
of the hardware out of kernel code and into a separate bit of data that
can be passed in to the kernel.

> It sounds interesting ... however, it also sounds like an area which
> might not impact the core kernel much ... or am I wrong about that?  The
> topics we're really looking for the Kernel Summit are ones that require
> cross system input and which can't simply be sorted out by organising an
> Embedded mini-summit.

One issue that does have wider impact is that the OpenFirmware bindings
can affect any driver level code - it means that drivers may need to
parse configuration out of the device tree as well as the mechanisms
they normally use.  This is already happening due to the current use but
will become more visible if more platforms adopt the device tree.  As
someone primarily working on driver/subsystem side stuff this is my
primary concern with expanded use of device tree - it's another set of
platform data code that needs writing in addition to the other schemes
currently in use.

On the other hand, if all the embedded architectures got together and
agreeded to move in this direction it'd be pretty much equivalent to
some new BIOS standard being introduced for PCs so perhaps not worth
worrying about at a general kernel level.

> Now if flattened device tree could help us out with BIOS, ACPI, EFI and
> the other myriad boot and identification standards that seem designed to
> hide system information rather than reveal it, then we might be all
> ears ...

Well, you could potentially try to render other BIOS data tables into
device tree format.  I'm not sure that the translation would be less
effort than parsing the existing data, though.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 11:48 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Tue, 2009-06-02 at 11:29 -0600, Grant Likely wrote:
>> One topic that seems to garner debate is the issue of decoupling the
>> kernel image from the target platform.  ie. On x86, PowerPC and Sparc
>> a kernel image will boot on any machine (assuming the needed drivers
>> are enabled), but this is rarely the case in embedded.  Most embedded
>> kernels require explicit board support selected at compile time with
>> no way to produce a generic kernel which will boot on a whole family
>> of devices, let alone the whole architecture.  Part of this is a
>> firmware issue, where existing firmware passes very little in the way
>> of hardware description to the kernel, but part is also not making
>> available any form of common language for describing the machine.
>
> OK, so my minimal understanding in this area lead me to believe this was
> because most embedded systems didn't have properly discoverable busses
> and therefore you had to tailor the kernel configuration exactly to the
> devices the system had.

Yes, mostly true.  The kernel must be explicitly told the layout of
the non-discoverable busses and interconnects.  One method is to use
per-machine statically compiled tables of platform devices, but
nothing forces embedded to do it that way...

>> I think that in the absence of any established standard like the PC
>> BIOS/EFI or a real Open Firmware interface, then the kernel should at
>> least offer a recommended interface so that multiplatform kernels are
>> possible without explicitly having the machine layout described to it
>> at compile time.  I know that some of the embedded distros are
>> interested in such a thing since it gets them away from shipping
>> separate images for each supported board.  ie. It's really hard to do
>> a generic live-cd without some form of multiplatform.  FDT is a great
>> approach, but it probably isn't the only option.  It would be worth
>> debating.
>
> It sounds interesting ... however, it also sounds like an area which
> might not impact the core kernel much ... or am I wrong about that?  The
> topics we're really looking for the Kernel Summit are ones that require
> cross system input and which can't simply be sorted out by organising an
> Embedded mini-summit.

Hmmm... in reading this thread and thinking about it more, I'm
beginning to think that it might really be a core kernel issue; or at
least a device driver policy one.  Regardless of architecture, at boot
time Linux must use some method to discover the system layout, be it:
1) Reading BIOS/EFI/ACPI/OpenFirmware/FDT data
2) Probing the bus (PCI, USB, etc)
3) Compiled into the kernel (tables of platform devices, machine specific code)

Many types of devices could be end up being discovered using any of
the above methods.  Ignoring for the time being the complexities and
history of the Linux UART drivers, I'm going to use 16550 serial ports
for an example.  On ARM, a platform device for an 16550 serial port
may be instantiated by machine specific init code, on PowerPC it will
be discovered by a device tree parser, on a PC it could be a legacy
port, and on all three it could hang off a PCI device.  The bus
connection and source of data are different in each case, but the same
core driver will handle all of them.  The real differences are in
discovery and decoding the configuration.

SPI devices (struct spi_device) is possibly a better example.
spi_device drivers that need additional configuration go looking at
the platform_data pointer in the struct device.  This is easy when the
device is hard coded into the kernel because the correct pdata struct
is initialized statically at build time.  When the device is
discovered via one of the other mechanisms, the question remains of
where should the code live that does the translation and fills in the
correct pdata?  The mmc_spi driver handles this by calling out to an
mmc_spi_get_pdata() function (drivers/spi/{mmc_spi,mmc_spi_of}.c.  If
running on an OF platform, mmc_spi_get_pdata() has the knowledge to
decode the device tree data and munge it into the pdata form needed by
the driver.  Both statically compiled and Device Tree described
mmc_spi configurations must be handled, and driver specific decode
methods must exist, but there I don't think there is any desire to
write multiple probe routines for each device driver.  The same issue
stands for i2c, MMIO, and other non-discoverable busses.

For drivers which require pdata, writing decode functions is
unavoidable, but it is unclear how to hook in that code with as little
impact on a device driver as possible.  To me the issue is, where
should that code live? and how should it get executed?  (which is why
I think it is a device driver policy issue)  I've used the example of
OF device tree bindings vs. static configuration, but it applies just
as readily to something like UEFI (ie. I see that ARM is a member of
the UEFI forum).  Here are some of my unsorted thoughts on the issue:

- Translation code is driver specific, so it should live as part of,
or in the vicinity of, the driver it works with
- My guess is that The Future(tm) will probably bring more methods of
describing machine configuration, not less;  It is worth debating now
about how to have multiple decoders for a single device driver.
- Devices on non-discoverable busses appear in both desktop and
embedded machines.  (sensors anyone?)  This is not just an embedded
issue.
- driver authors will probably implement only the decode methods that
they actually need.  It is likely that different people will develop
additional decoders.  These will need to co-exist peacefully.
- Some things are just hard and just require machine specific setup
code.  Things like weird CS selections on SPI busses, or clock
routing.  Decoding to pdata won't always be feasible and sometime
machine specific hooks must be used.  Need a method for machine setup
code to provide pdata.
- Binding algorithms are problematic.  Naming convention in the data
source won't always match Linux internal kernel naming so there must
be some logic for matching to the correct drivers.  Currently, an
exceptions table is used for i2c and spi busses (drivers/of/base.c:
of_modalias_table).  There aren't many entries in there now, but I'm
not sure it is a scalable solution in the long term.

And some possible approaches:
1) One option is to link a list of per-device decoder methods into the
kernel so that generic bus discovery code and call the correct pdata
decoder for each device when it is discovered.  Doing this ensures
that pdata is available before the driver's probe method is called and
completely isolates the driver code from the decode method.  However,
it also means that the entire list of decoders must be statically
linked into the kernel, and it is of no use at all to out-of-tree
drivers because it provides no method of linking in additional decode
hooks.  Many drivers used to do this in arch/powerpc, but we moved
away from it for the problems listed above.
2) It could be handled with wrapper drivers which have their own
struct device and get bound to data elements within the Device Tree or
other data source.  The wrapper driver could generate the pdata and
register a child platform device which gets bound to the generic
driver.  Doing it this way lets a decode method live in a module.
This works well for things which are currently struct platform_device,
and ensures that all data is available at driver probe time, but
doesn't fit well into the structure of other bus types (SPI, I2C,
MDIO, etc) without creating an indirection layer for decoding on each
bus.  It also means that for every real device, 2 struct device get
registered; one to bind against the decoder, and one to bind against
the real driver.  Maybe this isn't a significant memory consumption;
but it doesn't *feel* right to me.
3) The decoders could be linked into the drivers themselves.  The
mmc_spi driver uses this approach, though it is a bit crude, and the
mmc_spi_get_pdata routine must be modified for each.  I've been
thinking about the possibility of have a decoder function list
attached to the driver and use a common helper function to walk the
list until one of them is able to provide valid pdata.  This would
keep the decode method with the device driver (where it belongs
IMNSHO), but minimize the impact on the core of the driver as a whole
(only a function would be added).  But, this is still impact on the
driver which there may be resistance to.
4) write separate probe() routines for each type of discovery (static
pdata, device tree, etc).  Solves same problems as 3, but I think
results in more code and possibly a bunch of #ifdef'ry
5) other options I haven't thought of? ....

All of this options listed above have been talked about and
implemented to a lesser or greater degree without really coming to
much of a conclusion on how it should be approached and what the
impact on device drivers will be.  It is worth some debate.

> Now if flattened device tree could help us out with BIOS, ACPI, EFI and
> the other myriad boot and identification standards that seem designed to
> hide system information rather than reveal it, then we might be all
> ears ...

:-)

Interesting, but probably not much help here.  This would just be
translating (and imperfectly at that) from one machine representation
to another without a whole lot of benefit.  It is conceivable that
data sourced from multiple locations (probing, ACPI, EFI, and known
quirks) could be all funneled into a single FDT image and then that
data used for creating and registering device structures, but I don't
really see any benefit here.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Russell King]

On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
> Embedded PowerPC and Microblaze has tackled this problem with the
> "Flattened Device Tree" data format which is derived from the
> OpenFirmware specifications, and there is some interest and debate (as
> discussed recently on the ARM mailing list) about making flattened
> device tree usable by ARM also (which I'm currently
> proof-of-concepting).

Note that I have to point out that ARM will probably never be in a
situation where you can have a one kernel image boots on everything.
That _is_ practical today (and does happen with all PXA now) with what
we have within a very big restriction - which is that the kernel must
be built to support PXA and not Atmel SoCs.

I really don't think combining SoC support is going to be realistic,
device tree or not.  When we had just four machine types (RiscPC,
EBSA110, EBSA285, Netwinder) I did look into this and came to the
conclusion that it would be far too inefficient for there to be any
win.

The big problem we have is that the only commonality between different
SoCs is that the CPU executes ARM instructions.  Everything else is
entirely up to the SoC designer - eg location of memory, spacing of
memory banks, type of interrupt controller, etc is all highly SoC
specific.  Nothing outside of the ARM CPU itself is standardized.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: Representing Embedded Architectures at the Kernel Summit

[Bill Gatliff]

Russell King wrote:
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.
>
>   

And that diversity is precisely because of the diversity in ARM-based 
embedded platforms.

Such diversity means that kernel/driver development is a constant 
activity, which suggests that we shouldn't bother the effort to come up 
with a comprehensive solution because none will exist.  Rather, we 
should maintain and improve the ability to rapidly prototype and adapt.  
Things like furthering the deployment of platform_device, 
clocksource/clockdevice, and so on.

ARM diversity is already a LOT easier to deal with than it was under 
2.4, so we're making progress.


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: Representing Embedded Architectures at the Kernel Summit

[Bill Gatliff]

Russell King wrote:
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.
>
>   

And that diversity is precisely because of the diversity in ARM-based 
embedded platforms.

Such diversity means that kernel/driver development is a constant 
activity, which suggests that we shouldn't bother the effort to come up 
with a comprehensive solution because none will exist.  Rather, we 
should maintain and improve the ability to rapidly prototype and adapt.  
Things like furthering the deployment of platform_device, 
clocksource/clockdevice, and so on.

ARM diversity is already a LOT easier to deal with than it was under 
2.4, so we're making progress.


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 3:16 PM, Bill Gatliff <bgat@billgatliff.com> wrote:
> Russell King wrote:
>>
>> The big problem we have is that the only commonality between different
>> SoCs is that the CPU executes ARM instructions.  Everything else is
>> entirely up to the SoC designer - eg location of memory, spacing of
>> memory banks, type of interrupt controller, etc is all highly SoC
>> specific.  Nothing outside of the ARM CPU itself is standardized.
>
> And that diversity is precisely because of the diversity in ARM-based
> embedded platforms.
>
> Such diversity means that kernel/driver development is a constant activity,
> which suggests that we shouldn't bother the effort to come up with a
> comprehensive solution because none will exist.  Rather, we should maintain
> and improve the ability to rapidly prototype and adapt.  Things like
> furthering the deployment of platform_device, clocksource/clockdevice, and
> so on.

No, not comprehensive; just common.  It makes sense to spend the
effort on the patterns and devices which are common.  It may not cover
everything, but it doesn't have to to be valuable.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Bill Gatliff]

Russell King wrote:
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.
>
>   

And that diversity is precisely because of the diversity in ARM-based 
embedded platforms.

Such diversity means that kernel/driver development is a constant 
activity, which suggests that we shouldn't bother the effort to come up 
with a comprehensive solution because none will exist.  Rather, we 
should maintain and improve the ability to rapidly prototype and adapt.  
Things like furthering the deployment of platform_device, 
clocksource/clockdevice, and so on.

ARM diversity is already a LOT easier to deal with than it was under 
2.4, so we're making progress.


b.g.

-- 
Bill Gatliff
bgat@billgatliff.com

Re: Representing Embedded Architectures at the Kernel Summit

[Geert Uytterhoeven]

On Tue, Jun 2, 2009 at 23:10, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.
>
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

That sounds very similar to m68k, which does support generic kernels
(except for Sun-3, which uses a completely different MMU)?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Representing Embedded Architectures at the Kernel Summit

[Geert Uytterhoeven]

On Tue, Jun 2, 2009 at 23:10, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.
>
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

That sounds very similar to m68k, which does support generic kernels
(except for Sun-3, which uses a completely different MMU)?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Representing Embedded Architectures at the Kernel Summit

[Geert Uytterhoeven]

On Tue, Jun 2, 2009 at 23:10, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.
>
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

That sounds very similar to m68k, which does support generic kernels
(except for Sun-3, which uses a completely different MMU)?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Ralf Baechle]

On Tue, Jun 02, 2009 at 11:18:04PM +0200, Geert Uytterhoeven wrote:

> > The big problem we have is that the only commonality between different
> > SoCs is that the CPU executes ARM instructions.  Everything else is
> > entirely up to the SoC designer - eg location of memory, spacing of
> > memory banks, type of interrupt controller, etc is all highly SoC
> > specific.  Nothing outside of the ARM CPU itself is standardized.
> 
> That sounds very similar to m68k, which does support generic kernels
> (except for Sun-3, which uses a completely different MMU)?

For MIPS we currently have a concept of system families.  The user selects
one in the "System type" Kconfig menu.  With few exceptions that are not
worth considering these system types are incompatible so there are
currently 37 families of incompatible platforms with more being added.

  Ralf

Re: Representing Embedded Architectures at the Kernel Summit

[Robert Schwebel]

On Tue, Jun 02, 2009 at 10:10:58PM +0100, Russell King wrote:
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

And even with the cpu core, we usually build kernels with optimized
toolchains for their cpu family. But nevertheless - describing hardware
inside a mach* makes sense.

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: Representing Embedded Architectures at the Kernel Summit

[Robert Schwebel]

On Tue, Jun 02, 2009 at 10:10:58PM +0100, Russell King wrote:
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

And even with the cpu core, we usually build kernels with optimized
toolchains for their cpu family. But nevertheless - describing hardware
inside a mach* makes sense.

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: Representing Embedded Architectures at the Kernel Summit

[Robert Schwebel]

On Tue, Jun 02, 2009 at 10:10:58PM +0100, Russell King wrote:
> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

And even with the cpu core, we usually build kernels with optimized
toolchains for their cpu family. But nevertheless - describing hardware
inside a mach* makes sense.

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: Representing Embedded Architectures at the Kernel Summit

[Russell King]

On Tue, Jun 02, 2009 at 11:40:05PM +0200, Robert Schwebel wrote:
> And even with the cpu core, we usually build kernels with optimized
> toolchains for their cpu family.

I'm afraid that doesn't make a blind bit of difference to the kernel -
it passes the relevent optimization flags to gcc which tell the compiler
which instructions to use (based upon the CPU architecture) and how to
schedule those instructions (based upon the CPU type(s) chosen in the
kernel configuration.)

All that an "optimized" toolchain is is a toolchain with a different
default optimization selected.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 3:10 PM, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.

Agreed, and isn't really my position either.  I do understand that
some things, like different CPU cores requiring different code
generation options, will probably never be feasible to support in a
single image.  Others may be doable, but not worthwhile because of
memory constraints or impact on hot paths.

> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

I've been working on the Xilinx Virtex platform which is a CPU core
surrounded by a big FPGA fabric.  Doesn't get much non-standardized
that that.  :-)  It is trivial to build an image which boots on all of
the ppc440 platforms; the SoCs, and the Virtex5 FPGA.

Most of the 60x based SoCs have been merged too; all with very
different bus architectures, interrupt controllers and embedded
devices, but I do grant you that it isn't the same level of diversity
as in ARM land.

However, there has been no attempt to make a single powerpc image
which boots on everything; the cores are too different.  Instead, we
break it up by CPU type: 40x, 44x, 60x, 64 bit and the 8xx stuff that
everyone tries to forget.  IIUC, in ARM the MMU architecture is quite
varied, and that would have significant impact too.

Within each sub arch pretty much anything goes.  The kernel
initializes itself to the point of setting up the MMU and then probes
the machine definitions (arch/powerpc/platforms/*) until it finds one
which matches the device tree data.  At that point it is able to
branch off and set up the correct interrupt controller, register
devices and configure busses, etc.  As long as the correct machine
definition is compiled in, then everything goes.

As I'm sure you're about to point out :-), PowerPC machine probing
really isn't all that different from ARM machine probing except that
ARM uses the machine number and PowerPC uses the "model" property in
the device tree.  The real difference is the source of data describing
which devices are present; it just happens to be in an external data
structure instead of compiled in.  A lot of what was originally
thought to be machine specific code has just gone away because the
device tree infrastructure handles it elegantly.  Only the parts that
truly are machine specific continue to hang around.

I'm *not* suggesting that all ARM machines, or even most ARM machines
should be reworked to use something like the device tree.  What I am
suggesting is that there are many uses cases where it does make sense.
 For example, on the range of ARM netbooks rumored to come out near
the end of the year, the prospect of having a single kernel image boot
on multiple netbooks from multiple vendors is attractive to
distribution vendors like Canonical.  Device Tree certainly isn't the
only way to do this, but it is a viable one.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 3:10 PM, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.

Agreed, and isn't really my position either.  I do understand that
some things, like different CPU cores requiring different code
generation options, will probably never be feasible to support in a
single image.  Others may be doable, but not worthwhile because of
memory constraints or impact on hot paths.

> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

I've been working on the Xilinx Virtex platform which is a CPU core
surrounded by a big FPGA fabric.  Doesn't get much non-standardized
that that.  :-)  It is trivial to build an image which boots on all of
the ppc440 platforms; the SoCs, and the Virtex5 FPGA.

Most of the 60x based SoCs have been merged too; all with very
different bus architectures, interrupt controllers and embedded
devices, but I do grant you that it isn't the same level of diversity
as in ARM land.

However, there has been no attempt to make a single powerpc image
which boots on everything; the cores are too different.  Instead, we
break it up by CPU type: 40x, 44x, 60x, 64 bit and the 8xx stuff that
everyone tries to forget.  IIUC, in ARM the MMU architecture is quite
varied, and that would have significant impact too.

Within each sub arch pretty much anything goes.  The kernel
initializes itself to the point of setting up the MMU and then probes
the machine definitions (arch/powerpc/platforms/*) until it finds one
which matches the device tree data.  At that point it is able to
branch off and set up the correct interrupt controller, register
devices and configure busses, etc.  As long as the correct machine
definition is compiled in, then everything goes.

As I'm sure you're about to point out :-), PowerPC machine probing
really isn't all that different from ARM machine probing except that
ARM uses the machine number and PowerPC uses the "model" property in
the device tree.  The real difference is the source of data describing
which devices are present; it just happens to be in an external data
structure instead of compiled in.  A lot of what was originally
thought to be machine specific code has just gone away because the
device tree infrastructure handles it elegantly.  Only the parts that
truly are machine specific continue to hang around.

I'm *not* suggesting that all ARM machines, or even most ARM machines
should be reworked to use something like the device tree.  What I am
suggesting is that there are many uses cases where it does make sense.
 For example, on the range of ARM netbooks rumored to come out near
the end of the year, the prospect of having a single kernel image boot
on multiple netbooks from multiple vendors is attractive to
distribution vendors like Canonical.  Device Tree certainly isn't the
only way to do this, but it is a viable one.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 2, 2009 at 3:10 PM, Russell King <rmk@arm.linux.org.uk> wrote:
> On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:
>> Embedded PowerPC and Microblaze has tackled this problem with the
>> "Flattened Device Tree" data format which is derived from the
>> OpenFirmware specifications, and there is some interest and debate (as
>> discussed recently on the ARM mailing list) about making flattened
>> device tree usable by ARM also (which I'm currently
>> proof-of-concepting).
>
> Note that I have to point out that ARM will probably never be in a
> situation where you can have a one kernel image boots on everything.
> That _is_ practical today (and does happen with all PXA now) with what
> we have within a very big restriction - which is that the kernel must
> be built to support PXA and not Atmel SoCs.

Agreed, and isn't really my position either.  I do understand that
some things, like different CPU cores requiring different code
generation options, will probably never be feasible to support in a
single image.  Others may be doable, but not worthwhile because of
memory constraints or impact on hot paths.

> I really don't think combining SoC support is going to be realistic,
> device tree or not.  When we had just four machine types (RiscPC,
> EBSA110, EBSA285, Netwinder) I did look into this and came to the
> conclusion that it would be far too inefficient for there to be any
> win.
>
> The big problem we have is that the only commonality between different
> SoCs is that the CPU executes ARM instructions.  Everything else is
> entirely up to the SoC designer - eg location of memory, spacing of
> memory banks, type of interrupt controller, etc is all highly SoC
> specific.  Nothing outside of the ARM CPU itself is standardized.

I've been working on the Xilinx Virtex platform which is a CPU core
surrounded by a big FPGA fabric.  Doesn't get much non-standardized
that that.  :-)  It is trivial to build an image which boots on all of
the ppc440 platforms; the SoCs, and the Virtex5 FPGA.

Most of the 60x based SoCs have been merged too; all with very
different bus architectures, interrupt controllers and embedded
devices, but I do grant you that it isn't the same level of diversity
as in ARM land.

However, there has been no attempt to make a single powerpc image
which boots on everything; the cores are too different.  Instead, we
break it up by CPU type: 40x, 44x, 60x, 64 bit and the 8xx stuff that
everyone tries to forget.  IIUC, in ARM the MMU architecture is quite
varied, and that would have significant impact too.

Within each sub arch pretty much anything goes.  The kernel
initializes itself to the point of setting up the MMU and then probes
the machine definitions (arch/powerpc/platforms/*) until it finds one
which matches the device tree data.  At that point it is able to
branch off and set up the correct interrupt controller, register
devices and configure busses, etc.  As long as the correct machine
definition is compiled in, then everything goes.

As I'm sure you're about to point out :-), PowerPC machine probing
really isn't all that different from ARM machine probing except that
ARM uses the machine number and PowerPC uses the "model" property in
the device tree.  The real difference is the source of data describing
which devices are present; it just happens to be in an external data
structure instead of compiled in.  A lot of what was originally
thought to be machine specific code has just gone away because the
device tree infrastructure handles it elegantly.  Only the parts that
truly are machine specific continue to hang around.

I'm *not* suggesting that all ARM machines, or even most ARM machines
should be reworked to use something like the device tree.  What I am
suggesting is that there are many uses cases where it does make sense.
 For example, on the range of ARM netbooks rumored to come out near
the end of the year, the prospect of having a single kernel image boot
on multiple netbooks from multiple vendors is attractive to
distribution vendors like Canonical.  Device Tree certainly isn't the
only way to do this, but it is a viable one.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Wed, Jun 3, 2009 at 3:08 AM, Steve Langstaff
<steve.langstaff@pebblebay.com> wrote:
>> From: linux-embedded-owner@vger.kernel.org [mailto:linux-embedded-
>> owner@vger.kernel.org] On Behalf Of Russell King
>
>> The big problem we have is that the only commonality between different
>> SoCs is that the CPU executes ARM instructions.  Everything else is
>> entirely up to the SoC designer - eg location of memory, spacing of
>> memory banks, type of interrupt controller, etc is all highly SoC
>> specific.  Nothing outside of the ARM CPU itself is standardized.
>
> To my naive ears it sounds like this problem is crying out for ARM and the
> SoC designers to add a standardized "autoprobe" interface to the core to
> allow discovery of machine type and/or "location of memory, spacing of
> memory banks, type of interrupt controller, etc".
>
> The benefits of such mechanisms are well known, but what are the drawbacks?

Local bus probing probably imposes a lot of assumptions on a bus
designed to be as open as possible.  How are chip selects wired up?
What base addresses do devices respond to?  How do you know what the
device is?  What IRQ lines are used?  PCI solves this by exporting
configuration space which defines all of this, but PCI is considerably
more complex and not as fast as a CPU's local bus.  Similarly busses
like spi and i2c either have no probing protocol defined (spi), or
cannot always be reliably probed (i2c).

In short, the drawbacks are complexity on devices which cannot afford
the complexity.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Ralf Baechle]

On Tue, Jun 02, 2009 at 11:29:46AM -0600, Grant Likely wrote:

> One topic that seems to garner debate is the issue of decoupling the
> kernel image from the target platform.  ie. On x86, PowerPC and Sparc
> a kernel image will boot on any machine (assuming the needed drivers
> are enabled), but this is rarely the case in embedded.  Most embedded
> kernels require explicit board support selected at compile time with
> no way to produce a generic kernel which will boot on a whole family
> of devices, let alone the whole architecture.  Part of this is a
> firmware issue, where existing firmware passes very little in the way
> of hardware description to the kernel, but part is also not making
> available any form of common language for describing the machine.

Hardware is simple to incompatible to allow the generation of a single
"one size fits all" image.  To list a few reasons from the MIPS world:

  o little vs. big endian
  o 32-bit vs. 64-bit
  o different system firmwares
  o processors and peripherals requiring creative workarounds which for
    code size or performance reasons want to limit to those systems
    suffering from the issue.
  o often claustrophobically small memory sizes.
  o exactly no communality across all systems except the processor
    architecture.
  o vendors coming up with their own instruction set enhancements not
    supported by any competitor and insisting on their use for the extra
    bit of performance and product differenciation.
  o many users have a long standing "roll your own" attitude.
  o peripherals that are even less probeable than ISA cards

"Flattened Device Tree" can tackle only a small part of this but its a
step.

> Embedded PowerPC and Microblaze has tackled this problem with the
> "Flattened Device Tree" data format which is derived from the
> OpenFirmware specifications, and there is some interest and debate (as
> discussed recently on the ARM mailing list) about making flattened
> device tree usable by ARM also (which I'm currently
> proof-of-concepting).  Josh Boyer has already touched on discussing
> flattened device tree support at kernel summit in an email to the
> ksummit list last week (quoted below), and I'm wondering if a broader
> discussing would be warranted.

Agreed.

  Ralf

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Catalin Marinas]

Hi,

On Tue, 2009-06-02 at 15:22 +0000, James Bottomley wrote:
> So what we're looking for is a proposal to discuss the issues
> most affecting embedded architectures, or preview any features affecting
> the main kernel which embedded architectures might need ... or any other
> topics from embedded architectures which might need discussion or
> debate.

Some issues that come up on embedded systems (and not only):

      * Multiple coherency domains for devices - the system may have
        multiple bus levels, coherency ports, cache levels etc. Some
        devices in the system (but not all) may be able to "see" various
        cache levels but the DMA API (at least on ARM) cannot handle
        this. It may be useful to discuss how other embedded
        architectures handle this and come up with a unified solution

      * Better support for coherent DMA mask - currently ZONE_DMA is
        assumed to be in the bottom part of the memory which isn't
        always the case. Enabling NUMA may help but it is overkill for
        some systems. As above, a more unified solution across
        architectures would help

      * PIO block devices and non-coherent hardware - code like mpage.c
        assumes that the either the hardware is coherent or the device
        driver performs the cache flushing. The latter is true for
        DMA-capable device but not for PIO. The issue becomes visible
        with write-allocate caches and the device driver may not have
        the struct page information to call flush_dcache_page(). A
        proposed solution on the ARM lists was to differentiate (via
        some flags) between PIO and DMA block devices and use this
        information in mpage.c

      * Mixed endianness devices in the same system - this may only need
        dedicated readl_be/writel_be etc. macros but it could also be
        done by having bus-aware readl/writel-like macros

      * Asymmetric MP:
              * Different CPU frequencies
              * Different CPU features (e.g. floating point only one
                some CPUs): scheduler awareness, per-CPU hwcap bits (in
                case user space wants to set the affinity) 
              * Asymmetric workload balancing for power consumption (may
                be better to load 1 CPU at 60% than 4 at 15%) 

Thanks.

-- 
Catalin

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Josh Boyer]

On Wed, Jun 03, 2009 at 02:04:46PM +0100, Catalin Marinas wrote:
>      * Mixed endianness devices in the same system - this may only need
>        dedicated readl_be/writel_be etc. macros but it could also be
>        done by having bus-aware readl/writel-like macros

ioread/iowrite{8,16,32} and ioread/iowrite{8,16,32}_be don't suffice here?

josh

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Catalin Marinas]

On Wed, 2009-06-03 at 09:18 -0400, Josh Boyer wrote:
> On Wed, Jun 03, 2009 at 02:04:46PM +0100, Catalin Marinas wrote:
> >      * Mixed endianness devices in the same system - this may only need
> >        dedicated readl_be/writel_be etc. macros but it could also be
> >        done by having bus-aware readl/writel-like macros
> 
> ioread/iowrite{8,16,32} and ioread/iowrite{8,16,32}_be don't suffice here?

Yes, but there there are many drivers that only use readl/writel (and
arch/arm makes the assumption, maybe correctly, that this is little
endian only).

I think that's useful even if the outcome of such discussion is better
documentation on the above functions/macros (grepping Documentation/
doesn't show any reference).

Thanks.

-- 
Catalin

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Josh Boyer]

On Wed, Jun 03, 2009 at 02:45:37PM +0100, Catalin Marinas wrote:
>On Wed, 2009-06-03 at 09:18 -0400, Josh Boyer wrote:
>> On Wed, Jun 03, 2009 at 02:04:46PM +0100, Catalin Marinas wrote:
>> >      * Mixed endianness devices in the same system - this may only need
>> >        dedicated readl_be/writel_be etc. macros but it could also be
>> >        done by having bus-aware readl/writel-like macros
>> 
>> ioread/iowrite{8,16,32} and ioread/iowrite{8,16,32}_be don't suffice here?
>
>Yes, but there there are many drivers that only use readl/writel (and
>arch/arm makes the assumption, maybe correctly, that this is little
>endian only).

readl/writel are little-endian only.

>I think that's useful even if the outcome of such discussion is better
>documentation on the above functions/macros (grepping Documentation/
>doesn't show any reference).

I think we could perhaps start with just writting some of that documentation
and trying to get it into the kernel.  I'm not sure this specific item is
really worth of a KS topic.

josh

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Jean-Christophe PLAGNIOL-VILLARD]

> 
>       * Asymmetric MP:
>               * Different CPU frequencies
>               * Different CPU features (e.g. floating point only one
>                 some CPUs): scheduler awareness, per-CPU hwcap bits (in
>                 case user space wants to set the affinity) 
>               * Asymmetric workload balancing for power consumption (may
>                 be better to load 1 CPU at 60% than 4 at 15%) 
I'll add
		* Different Core ARCH
		* FDT or similar to describe I/O (MEM, PCI, GPIO) acessible for
		  each instance
		* Mailbox Architecture
		* boot preocedure (bootloader as example done by Kumar Gala
		  for the mpc8572ds in linux & U-Boot)
		* sharing rootfs (RO) (reduce the rootfs size on embedded)

Best Regards,
J.

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Wed, 2009-06-03 at 14:04 +0100, Catalin Marinas wrote:
> Hi,
> 
> On Tue, 2009-06-02 at 15:22 +0000, James Bottomley wrote:
> > So what we're looking for is a proposal to discuss the issues
> > most affecting embedded architectures, or preview any features affecting
> > the main kernel which embedded architectures might need ... or any other
> > topics from embedded architectures which might need discussion or
> > debate.
> 
> Some issues that come up on embedded systems (and not only):
> 
>       * Multiple coherency domains for devices - the system may have
>         multiple bus levels, coherency ports, cache levels etc. Some
>         devices in the system (but not all) may be able to "see" various
>         cache levels but the DMA API (at least on ARM) cannot handle
>         this. It may be useful to discuss how other embedded
>         architectures handle this and come up with a unified solution

So this is partially what the dma_sync_for_{device|cpu} is supposed to
be helping with.  By and large, the DMA API tries to hide the
complexities of coherency domains from the user.  The actual API, as far
as it goes, seems to do this OK.  We have synchronisation issues that
mmiowb() and friends help with ... what's the actual problem here?

>       * Better support for coherent DMA mask - currently ZONE_DMA is
>         assumed to be in the bottom part of the memory which isn't
>         always the case. Enabling NUMA may help but it is overkill for
>         some systems. As above, a more unified solution across
>         architectures would help

So ZONE_DMA and coherent memory allocation as represented by the
coherent mask are really totally separate things.  The idea of ZONE_DMA
was really that if you had an ISA device, allocations from ZONE_DMA
would be able to access the allocated memory without bouncing.  Since
ISA is really going away, this definition has been hijacked.  If your
problem is just that you need memory allocated on a certain physical
mask and neither GFP_DMA or GFP_DMA32 cut it for you, then we could
revisit the kmalloc_mask() proposal again ... but the consensus last
time was that no-one really had a compelling use case that couldn't be
covered by GFP_DMA32.

>       * PIO block devices and non-coherent hardware - code like mpage.c
>         assumes that the either the hardware is coherent or the device
>         driver performs the cache flushing. The latter is true for
>         DMA-capable device but not for PIO. The issue becomes visible
>         with write-allocate caches and the device driver may not have
>         the struct page information to call flush_dcache_page(). A
>         proposed solution on the ARM lists was to differentiate (via
>         some flags) between PIO and DMA block devices and use this
>         information in mpage.c

flush_dcache_page() is supposed to be for making the data visible to the
user ... that coherency is supposed to be managed by the block layer.
The DMA API is specifically aimed at device to kernel space
coherency ... although if you line up all your aliases, that can also be
device to userspace.  Technically though we have two separate APIs for
user<->kernel coherency and device<->kernel coherency.  What's the path
you're seeing this problem down?  SG_IO to a device doing PIO should be
handling this correctly.

>       * Mixed endianness devices in the same system - this may only need
>         dedicated readl_be/writel_be etc. macros but it could also be
>         done by having bus-aware readl/writel-like macros

We have ioreadXbe for this exact case (similar problem on parisc)

>       * Asymmetric MP:
>               * Different CPU frequencies
>               * Different CPU features (e.g. floating point only one
>                 some CPUs): scheduler awareness, per-CPU hwcap bits (in
>                 case user space wants to set the affinity) 
>               * Asymmetric workload balancing for power consumption (may
>                 be better to load 1 CPU at 60% than 4 at 15%) 

This actually just works(tm) for me on a voyager system running SMP with
a mixed 486/586 set of processors ... what's the problem?  The only
issue I see is that you have to set the capabilities of the boot CPU to
the intersection of the mixture otherwise setup goes wrong, but
otherwise it seems to work OK.

James

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Russell King]

On Wed, Jun 03, 2009 at 12:19:57PM -0400, James Bottomley wrote:
> On Wed, 2009-06-03 at 14:04 +0100, Catalin Marinas wrote:
> >       * Better support for coherent DMA mask - currently ZONE_DMA is
> >         assumed to be in the bottom part of the memory which isn't
> >         always the case. Enabling NUMA may help but it is overkill for
> >         some systems. As above, a more unified solution across
> >         architectures would help
> 
> So ZONE_DMA and coherent memory allocation as represented by the
> coherent mask are really totally separate things.  The idea of ZONE_DMA
> was really that if you had an ISA device, allocations from ZONE_DMA
> would be able to access the allocated memory without bouncing.  Since
> ISA is really going away, this definition has been hijacked.  If your
> problem is just that you need memory allocated on a certain physical
> mask and neither GFP_DMA or GFP_DMA32 cut it for you, then we could
> revisit the kmalloc_mask() proposal again ... but the consensus last
> time was that no-one really had a compelling use case that couldn't be
> covered by GFP_DMA32.

I'm not aware of such a discussion; I keep running into issues here.  In
fact, on ARM the DMA mask is exactly that - it's a 100% proper mask.  It's
not a bunch of zeros in the MSB followed by a bunch of ones down to the
LSB.  It can be a bunch of ones, a bunch of zeros, followed by a bunch of
ones.

The way we occasionally have to deal with this is to trial an allocation,
see if the physical address fits, if not free the page and try again with
GFP_DMA set.

We do certain checks on the DMA mask - notably that a GFP_DMA allocation
will satisfy the mask which has been passed.

I've never submitted the patch which does this in the ARM coherent DMA
allocator, but it's something that occasionally crops up as being
necessary - I've always thought the allocate-by-mask stuff would
eventually be merged.

> >       * PIO block devices and non-coherent hardware - code like mpage.c
> >         assumes that the either the hardware is coherent or the device
> >         driver performs the cache flushing. The latter is true for
> >         DMA-capable device but not for PIO. The issue becomes visible
> >         with write-allocate caches and the device driver may not have
> >         the struct page information to call flush_dcache_page(). A
> >         proposed solution on the ARM lists was to differentiate (via
> >         some flags) between PIO and DMA block devices and use this
> >         information in mpage.c
> 
> flush_dcache_page() is supposed to be for making the data visible to the
> user ... that coherency is supposed to be managed by the block layer.
> The DMA API is specifically aimed at device to kernel space
> coherency ... although if you line up all your aliases, that can also be
> device to userspace.  Technically though we have two separate APIs for
> user<->kernel coherency and device<->kernel coherency.  What's the path
> you're seeing this problem down?  SG_IO to a device doing PIO should be
> handling this correctly.

There's many stories I've heard on what is supposed to take care of the
coherency that I now just close my ears to the problem and chant "it
doesn't exist, people aren't seeing it, mainline folk just don't give
a damn".  Really.  It is a problem on _some_ ARM devices and has been
for several years now, and I've 100% given up caring about it.

So people who see the problem just have to suffer with it, and they have
to accept that the Linux kernel sucks with PIO on ARM hardware.

Unless they use a driver I've written which has the necessary callbacks
in it to ensure cache coherency (like MMC).  IDE... forget it.

Yes, that taste you're experiencing is my bitterness on this subject.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Andrew Morton]

On Wed, 3 Jun 2009 18:09:25 +0100
Russell King <rmk@arm.linux.org.uk> wrote:

> In
> fact, on ARM the DMA mask is exactly that - it's a 100% proper mask.  It's
> not a bunch of zeros in the MSB followed by a bunch of ones down to the
> LSB.  It can be a bunch of ones, a bunch of zeros, followed by a bunch of
> ones.
> 
> The way we occasionally have to deal with this is to trial an allocation,
> see if the physical address fits, if not free the page and try again with
> GFP_DMA set.

A couple of times I've suggested that we have the ability to allocate
one zone per address bit, so a 32-bit machine with 4k pages would end
up having 20 zones.  Then, your funny DMA mask can be directly passed
into the page allocator as a zone mask and voila, I think.

> There's many stories I've heard on what is supposed to take care of the
> coherency that I now just close my ears to the problem and chant "it
> doesn't exist, people aren't seeing it, mainline folk just don't give
> a damn".  Really.  It is a problem on _some_ ARM devices and has been
> for several years now, and I've 100% given up caring about it.

I wasn't even aware that there was an issue here.  Please don't blame
"mainline folk" for something they weren't told about!

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[James Bottomley]

On Wed, 2009-06-03 at 11:43 -0700, Andrew Morton wrote:
> On Wed, 3 Jun 2009 18:09:25 +0100
> Russell King <rmk@arm.linux.org.uk> wrote:
> 
> > In
> > fact, on ARM the DMA mask is exactly that - it's a 100% proper mask.  It's
> > not a bunch of zeros in the MSB followed by a bunch of ones down to the
> > LSB.  It can be a bunch of ones, a bunch of zeros, followed by a bunch of
> > ones.
> > 
> > The way we occasionally have to deal with this is to trial an allocation,
> > see if the physical address fits, if not free the page and try again with
> > GFP_DMA set.
> 
> A couple of times I've suggested that we have the ability to allocate
> one zone per address bit, so a 32-bit machine with 4k pages would end
> up having 20 zones.  Then, your funny DMA mask can be directly passed
> into the page allocator as a zone mask and voila, I think.

The objection I heard to that one is that the zone machinery works
better with fewer zones ... but we could certainly align them along
known boundaries for allocations (if it's only bit X that's the problem,
say, you only need an additional zone covering that one).

Based on this, I dug up the initial proposal, it was the Ottawa Kernel
Summit in 2005 (I'm a packrat; I keep all my old presentations):

http://www.hansenpartnership.com/sites/hansenpartnership.com/files/jejb/kernel_summit_iommu.pdf

kmalloc_mask() is right at the end.  It basically died for lack of
interest and the fact that GFP_DMA32 satisfied 99% of the actual use
cases.

James

RE: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[David VomLehn (dvomlehn)]

David Delaney has a proof-of-concept of an idea of his which was
presented at the last CELF, which is basically to put the kernel and
loadable kernel modules closely enough together that you can avoid the
use of long jumps. He sees a better than 1% improvement in performance,
which we've duplicated using a slightly different approach. This is nice
payback for little work and, though it doesn't help on all processors,
it helps on several.

The problem is: how do you allocate memory with the magical "close to
the kernel" attribute? We have something that adds a new ZONE_KERNEL
(this name has some problems, actually). It seems like a pretty good
solution if you look at zones as conceptually concentric usages, but
with the current zone implementation, each zone must be contiguous. So,
if we're talking about changing what zones are done, I'd like to throw
this into the pot.

> -----Original Message-----
> From: linux-embedded-owner@vger.kernel.org 
> [mailto:linux-embedded-owner@vger.kernel.org] On Behalf Of 
> Andrew Morton
> Sent: Wednesday, June 03, 2009 11:44 AM
> To: Russell King
> Cc: James.Bottomley@HansenPartnership.com; 
> linux-arch@vger.kernel.org; linux-embedded@vger.kernel.org; 
> ksummit-2009-discuss@lists.linux-foundation.org
> Subject: Re: [Ksummit-2009-discuss] Representing Embedded 
> Architectures at the Kernel Summit
> 
> On Wed, 3 Jun 2009 18:09:25 +0100
> Russell King <rmk@arm.linux.org.uk> wrote:
> 
> > In
> > fact, on ARM the DMA mask is exactly that - it's a 100% 
> proper mask.  It's
> > not a bunch of zeros in the MSB followed by a bunch of ones 
> down to the
> > LSB.  It can be a bunch of ones, a bunch of zeros, followed 
> by a bunch of
> > ones.
> > 
> > The way we occasionally have to deal with this is to trial 
> an allocation,
> > see if the physical address fits, if not free the page and 
> try again with
> > GFP_DMA set.
> 
> A couple of times I've suggested that we have the ability to allocate
> one zone per address bit, so a 32-bit machine with 4k pages would end
> up having 20 zones.  Then, your funny DMA mask can be directly passed
> into the page allocator as a zone mask and voila, I think.
> 
> > There's many stories I've heard on what is supposed to take 
> care of the
> > coherency that I now just close my ears to the problem and chant "it
> > doesn't exist, people aren't seeing it, mainline folk just 
> don't give
> > a damn".  Really.  It is a problem on _some_ ARM devices 
> and has been
> > for several years now, and I've 100% given up caring about it.
> 
> I wasn't even aware that there was an issue here.  Please don't blame
> "mainline folk" for something they weren't told about!
> 
> --
> To unsubscribe from this list: send the line "unsubscribe 
> linux-embedded" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Mike Frysinger]

On Wed, Jun 3, 2009 at 23:11, David VomLehn (dvomlehn) wrote:
> David Delaney has a proof-of-concept of an idea of his which was
> presented at the last CELF, which is basically to put the kernel and
> loadable kernel modules closely enough together that you can avoid the
> use of long jumps. He sees a better than 1% improvement in performance,
> which we've duplicated using a slightly different approach. This is nice
> payback for little work and, though it doesn't help on all processors,
> it helps on several.

it would help on the Blackfin architecture.  we compile all kernel
modules with -mlong-call because of this issue.

> The problem is: how do you allocate memory with the magical "close to
> the kernel" attribute? We have something that adds a new ZONE_KERNEL
> (this name has some problems, actually). It seems like a pretty good
> solution if you look at zones as conceptually concentric usages, but
> with the current zone implementation, each zone must be contiguous. So,
> if we're talking about changing what zones are done, I'd like to throw
> this into the pot.

what do you do if the alloc fails ?  return back to userspace with
something like ENOMEM and have it retry with a module that was
compiled with -mlong-call ?
-mike

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Mike Frysinger]

On Wed, Jun 3, 2009 at 23:11, David VomLehn (dvomlehn) wrote:
> David Delaney has a proof-of-concept of an idea of his which was
> presented at the last CELF, which is basically to put the kernel and
> loadable kernel modules closely enough together that you can avoid the
> use of long jumps. He sees a better than 1% improvement in performance,
> which we've duplicated using a slightly different approach. This is nice
> payback for little work and, though it doesn't help on all processors,
> it helps on several.

it would help on the Blackfin architecture.  we compile all kernel
modules with -mlong-call because of this issue.

> The problem is: how do you allocate memory with the magical "close to
> the kernel" attribute? We have something that adds a new ZONE_KERNEL
> (this name has some problems, actually). It seems like a pretty good
> solution if you look at zones as conceptually concentric usages, but
> with the current zone implementation, each zone must be contiguous. So,
> if we're talking about changing what zones are done, I'd like to throw
> this into the pot.

what do you do if the alloc fails ?  return back to userspace with
something like ENOMEM and have it retry with a module that was
compiled with -mlong-call ?
-mike

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Mel Gorman]

On Wed, Jun 03, 2009 at 11:11:13PM -0400, David VomLehn (dvomlehn) wrote:
> David Delaney has a proof-of-concept of an idea of his which was
> presented at the last CELF, which is basically to put the kernel and
> loadable kernel modules closely enough together that you can avoid the
> use of long jumps. He sees a better than 1% improvement in performance,
> which we've duplicated using a slightly different approach. This is nice
> payback for little work and, though it doesn't help on all processors,
> it helps on several.
> 
> The problem is: how do you allocate memory with the magical "close to
> the kernel" attribute? We have something that adds a new ZONE_KERNEL
> (this name has some problems, actually).

As this is about addresses of text, I imagine that you really care about the
virtual address the module is loaded at which is what the virtual address
space is responsible for and not the physical addresses which zones are
concerned with. Would that be right?

If that was the case, this could be potentially be done by moving where
the vmalloc address space is located or possibly splitting it in two. By
locating some portion of the vmalloc address space above the kernel
image, the kernel modules could be loaded in there.

It's different for the DMA problem, it really requires particular physical
addresses. No one trying to implementat Andrew's suggestion is a bit of a
surprise because basically, it'd do the job as far as I can see but is not
an issue that hurts me so I never sat down to try implementing it. Granted,
increasing the number of zones adds its own problem but it's for large numbers
of zones and there are other things that could be done to the allocator to
reduce its cache footprint. The big plus is that it plays very well with
reclaim and I would expect it to perform better than than searching the
free lists for a suitable page which would be a bit of a hatchet job.

> It seems like a pretty good
> solution if you look at zones as conceptually concentric usages, but
> with the current zone implementation, each zone must be contiguous. So,
> if we're talking about changing what zones are done, I'd like to throw
> this into the pot.
> 
> > -----Original Message-----
> > From: linux-embedded-owner@vger.kernel.org 
> > [mailto:linux-embedded-owner@vger.kernel.org] On Behalf Of 
> > Andrew Morton
> > Sent: Wednesday, June 03, 2009 11:44 AM
> > To: Russell King
> > Cc: James.Bottomley@HansenPartnership.com; 
> > linux-arch@vger.kernel.org; linux-embedded@vger.kernel.org; 
> > ksummit-2009-discuss@lists.linux-foundation.org
> > Subject: Re: [Ksummit-2009-discuss] Representing Embedded 
> > Architectures at the Kernel Summit
> > 
> > On Wed, 3 Jun 2009 18:09:25 +0100
> > Russell King <rmk@arm.linux.org.uk> wrote:
> > 
> > > In
> > > fact, on ARM the DMA mask is exactly that - it's a 100% 
> > proper mask.  It's
> > > not a bunch of zeros in the MSB followed by a bunch of ones 
> > down to the
> > > LSB.  It can be a bunch of ones, a bunch of zeros, followed 
> > by a bunch of
> > > ones.
> > > 
> > > The way we occasionally have to deal with this is to trial 
> > an allocation,
> > > see if the physical address fits, if not free the page and 
> > try again with
> > > GFP_DMA set.
> > 
> > A couple of times I've suggested that we have the ability to allocate
> > one zone per address bit, so a 32-bit machine with 4k pages would end
> > up having 20 zones.  Then, your funny DMA mask can be directly passed
> > into the page allocator as a zone mask and voila, I think.
> > 
> > > There's many stories I've heard on what is supposed to take 
> > care of the
> > > coherency that I now just close my ears to the problem and chant "it
> > > doesn't exist, people aren't seeing it, mainline folk just 
> > don't give
> > > a damn".  Really.  It is a problem on _some_ ARM devices 
> > and has been
> > > for several years now, and I've 100% given up caring about it.
> > 
> > I wasn't even aware that there was an issue here.  Please don't blame
> > "mainline folk" for something they weren't told about!
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe 
> > linux-embedded" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> _______________________________________________
> Ksummit-2009-discuss mailing list
> Ksummit-2009-discuss@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/ksummit-2009-discuss
> 

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Catalin Marinas]

On Wed, 2009-06-03 at 12:19 -0400, James Bottomley wrote:
> On Wed, 2009-06-03 at 14:04 +0100, Catalin Marinas wrote:
> > On Tue, 2009-06-02 at 15:22 +0000, James Bottomley wrote:
> > > So what we're looking for is a proposal to discuss the issues
> > > most affecting embedded architectures, or preview any features affecting
> > > the main kernel which embedded architectures might need ... or any other
> > > topics from embedded architectures which might need discussion or
> > > debate.
> > 
> > Some issues that come up on embedded systems (and not only):
> > 
> >       * Multiple coherency domains for devices - the system may have
> >         multiple bus levels, coherency ports, cache levels etc. Some
> >         devices in the system (but not all) may be able to "see" various
> >         cache levels but the DMA API (at least on ARM) cannot handle
> >         this. It may be useful to discuss how other embedded
> >         architectures handle this and come up with a unified solution
> 
> So this is partially what the dma_sync_for_{device|cpu} is supposed to
> be helping with.  By and large, the DMA API tries to hide the
> complexities of coherency domains from the user.  The actual API, as far
> as it goes, seems to do this OK.

Yes, the dma_sync_* API is probably OK. The actual implementation should
become aware of various coherency domains on the same system (it could
hold this information in one of the bus-related structures). Currently,
devices that can access the CPU (inner or outer) cache have drivers
modified to avoid calling the dma_sync_* functions (since other devices
need such functions).

If other embedded architectures face similar issues, it is worth
discussing and maybe come up with a common solution (of course, like
most topics, they could simply be discussed on the mailing lists rather
than at the KS).

> >       * Better support for coherent DMA mask - currently ZONE_DMA is
> >         assumed to be in the bottom part of the memory which isn't
> >         always the case. Enabling NUMA may help but it is overkill for
> >         some systems. As above, a more unified solution across
> >         architectures would help
> 
> So ZONE_DMA and coherent memory allocation as represented by the
> coherent mask are really totally separate things.  The idea of ZONE_DMA
> was really that if you had an ISA device, allocations from ZONE_DMA
> would be able to access the allocated memory without bouncing.  Since
> ISA is really going away, this definition has been hijacked.  If your
> problem is just that you need memory allocated on a certain physical
> mask and neither GFP_DMA or GFP_DMA32 cut it for you, then we could
> revisit the kmalloc_mask() proposal again ... but the consensus last
> time was that no-one really had a compelling use case that couldn't be
> covered by GFP_DMA32.

Russell already commented on this. As an example, I have a platform with
two blocks of RAM - 512MB @ 0x20000000 and 512MB @ 0x70000000 - but only
the higher one allows DMA.

> >       * PIO block devices and non-coherent hardware - code like mpage.c
> >         assumes that the either the hardware is coherent or the device
> >         driver performs the cache flushing. The latter is true for
> >         DMA-capable device but not for PIO. The issue becomes visible
> >         with write-allocate caches and the device driver may not have
> >         the struct page information to call flush_dcache_page(). A
> >         proposed solution on the ARM lists was to differentiate (via
> >         some flags) between PIO and DMA block devices and use this
> >         information in mpage.c
> 
> flush_dcache_page() is supposed to be for making the data visible to the
> user ... that coherency is supposed to be managed by the block layer.

I'm referring to kernel<->user coherency issues and yes,
flush_dcache_page() is the function supposed to handle this. It's only
that it isn't always called in the block or VFS layers (for example, to
be able to use ext2 over compact flash using pata I had to add a hack so
that flush_dcache_page is called from mpage_end_io_read).

Some devices like Russell's mmci.c use scatter lists and they have
access to the page structure and perform the flushing. I noticed that
for some block devices you can't easily retrieve the page structure (I
would need to check the code for more precise references). But if the
driver is somehow marked as PIO, the VFS layer could ensure the
coherency.

> >       * Mixed endianness devices in the same system - this may only need
> >         dedicated readl_be/writel_be etc. macros but it could also be
> >         done by having bus-aware readl/writel-like macros
> 
> We have ioreadXbe for this exact case (similar problem on parisc)

OK, probably not worth a new topic. As it was mentioned on
linux-embedded already, it may just need better documention (there is no
reference to ioread* in Documentation/ and most devices seem to use
readl/writel etc.).

> >       * Asymmetric MP:
> >               * Different CPU frequencies
> >               * Different CPU features (e.g. floating point only one
> >                 some CPUs): scheduler awareness, per-CPU hwcap bits (in
> >                 case user space wants to set the affinity) 
> >               * Asymmetric workload balancing for power consumption (may
> >                 be better to load 1 CPU at 60% than 4 at 15%) 
> 
> This actually just works(tm) for me on a voyager system running SMP with
> a mixed 486/586 set of processors ... what's the problem?  The only
> issue I see is that you have to set the capabilities of the boot CPU to
> the intersection of the mixture otherwise setup goes wrong, but
> otherwise it seems to work OK.

You can set the capabilities to the intersection of the CPU features but
that's not the goal. We'll see multiprocessor systems with only one (out
of 2, 4 etc.) of the CPUs having some features (like media processing
instructions). That's the case on embedded where the number of gates is
limited and the battery saving is important but you want to use the
extra features and not limit them. So the code I currently have for such
configuration is to trap the undefined instructions and set the CPU
affinity to the faulty threads (the affinity could be reset after some
time). Could it be done better? I think that's worth discussing.

-- 
Catalin

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Thomas Petazzoni]

Le Wed, 03 Jun 2009 12:19:57 -0400,
James Bottomley <James.Bottomley@HansenPartnership.com> a écrit :

> So ZONE_DMA and coherent memory allocation as represented by the
> coherent mask are really totally separate things.  The idea of
> ZONE_DMA was really that if you had an ISA device, allocations from
> ZONE_DMA would be able to access the allocated memory without
> bouncing.  Since ISA is really going away, this definition has been
> hijacked.  If your problem is just that you need memory allocated on
> a certain physical mask and neither GFP_DMA or GFP_DMA32 cut it for
> you, then we could revisit the kmalloc_mask() proposal again ... but
> the consensus last time was that no-one really had a compelling use
> case that couldn't be covered by GFP_DMA32.

Back a few years ago, I was working on a MIPS platform which had 256 MB
of RAM attached to the CPU memory controller and 128 MB attached to an
external memory controller. The layout of the memories was: 256 MB
CPU-attached memory first, and then the 128 MB
external-controller-attached memory.

Now, back to the DMA discussion: the Ethernet controller, which was
part of that external controller also driving the 128 MB bank of
memory, could only DMA to and from memory controlled by that same
controller (i.e only to the *top* 128 MB of the physical address
space). I'm by far not an mm expert, but as far as I could understand
the zone mechanism, it was not possible to describe such a
physical memory configuration where DMA-able memory is only at the top.

In the end, I ended up passing mem=..., managing manually a few
megabytes of memory at the top of the physical address space, and
hacking the Ethernet driver to copy back and forth the skb contents
between the main memory and the DMA-reserved memory.

So when Calatalin Marinas says « currently ZONE_DMA is assumed to be in
the bottom part of the memory which isn't always the case », I cannot
agree more.

Reference:
http://www.linux-mips.org/archives/linux-mips/2004-09/msg00152.html

Sincerly,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Re: Representing Embedded Architectures at the Kernel Summit

[Mike Rapoport]

James Bottomley wrote:
> Hi All,
> 
> We've got to the point where there are simply too many embedded
> architectures to invite all the arch maintainers to the kernel summit.
> So, this year, we thought we'd do embedded via topic driven invitations
> instead.  So what we're looking for is a proposal to discuss the issues
> most affecting embedded architectures, or preview any features affecting
> the main kernel which embedded architectures might need ... or any other
> topics from embedded architectures which might need discussion or
> debate.

Another issue that affects embedded architectures is drivers initialization
order. There are a lot of cases when you need the drivers to be initialized in
particular order, and current initcalls scheme does not allow fine grained
control for it.

-- 
Sincerely yours,
Mike.

Re: Representing Embedded Architectures at the Kernel Summit

[Mike Frysinger]

On Tue, Jun 16, 2009 at 02:42, Mike Rapoport wrote:
> James Bottomley wrote:
>> We've got to the point where there are simply too many embedded
>> architectures to invite all the arch maintainers to the kernel summit.
>> So, this year, we thought we'd do embedded via topic driven invitations
>> instead.  So what we're looking for is a proposal to discuss the issues
>> most affecting embedded architectures, or preview any features affecting
>> the main kernel which embedded architectures might need ... or any other
>> topics from embedded architectures which might need discussion or
>> debate.
>
> Another issue that affects embedded architectures is drivers initialization
> order. There are a lot of cases when you need the drivers to be initialized in
> particular order, and current initcalls scheme does not allow fine grained
> control for it.

example: device configuration information stored in i2c eeprom (i.e.
dimensions of attached framebuffer), but i2c is not available when
framebuffer layer is setup.  framebuffer driver has to be built as a
module and loaded by userspace, or i2c information is read by
bootloader and passed down to the kernel.
-mike

Re: Representing Embedded Architectures at the Kernel Summit

[Mike Frysinger]

On Tue, Jun 16, 2009 at 02:42, Mike Rapoport wrote:
> James Bottomley wrote:
>> We've got to the point where there are simply too many embedded
>> architectures to invite all the arch maintainers to the kernel summit.
>> So, this year, we thought we'd do embedded via topic driven invitations
>> instead.  So what we're looking for is a proposal to discuss the issues
>> most affecting embedded architectures, or preview any features affecting
>> the main kernel which embedded architectures might need ... or any other
>> topics from embedded architectures which might need discussion or
>> debate.
>
> Another issue that affects embedded architectures is drivers initialization
> order. There are a lot of cases when you need the drivers to be initialized in
> particular order, and current initcalls scheme does not allow fine grained
> control for it.

example: device configuration information stored in i2c eeprom (i.e.
dimensions of attached framebuffer), but i2c is not available when
framebuffer layer is setup.  framebuffer driver has to be built as a
module and loaded by userspace, or i2c information is read by
bootloader and passed down to the kernel.
-mike

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Ralf Baechle]

On Tue, Jun 16, 2009 at 04:06:48AM -0400, Mike Frysinger wrote:

> On Tue, Jun 16, 2009 at 02:42, Mike Rapoport wrote:
> > James Bottomley wrote:
> >> We've got to the point where there are simply too many embedded
> >> architectures to invite all the arch maintainers to the kernel summit.
> >> So, this year, we thought we'd do embedded via topic driven invitations
> >> instead.  So what we're looking for is a proposal to discuss the issues
> >> most affecting embedded architectures, or preview any features affecting
> >> the main kernel which embedded architectures might need ... or any other
> >> topics from embedded architectures which might need discussion or
> >> debate.
> >
> > Another issue that affects embedded architectures is drivers initialization
> > order. There are a lot of cases when you need the drivers to be initialized in
> > particular order, and current initcalls scheme does not allow fine grained
> > control for it.
> 
> example: device configuration information stored in i2c eeprom (i.e.
> dimensions of attached framebuffer), but i2c is not available when
> framebuffer layer is setup.  framebuffer driver has to be built as a
> module and loaded by userspace, or i2c information is read by
> bootloader and passed down to the kernel.

I2C or similar busses can be a particularly annoying if they contain
essential configuration information such as memory size which is needed
long before anything else.  So for far a common solution is that platforms
are carrying a private (aka redundant, ugly) early-i2c system that's just
about sufficient for this purpose.

  Ralf

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[H. Peter Anvin]

Ralf Baechle wrote:
> 
> I2C or similar busses can be a particularly annoying if they contain
> essential configuration information such as memory size which is needed
> long before anything else.  So for far a common solution is that platforms
> are carrying a private (aka redundant, ugly) early-i2c system that's just
> about sufficient for this purpose.
> 

For what it's worth, this is true for pretty much ALL systems with
removable memory modules, since Serial Presence Detect (SPD) is
electrically equivalent to I2C.

However, on most systems, even embedded, bringing up memory falls on
firmware (sometimes in the form of a boot loader) so Linux rarely sees it.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[Ralf Baechle]

On Tue, Jun 16, 2009 at 09:26:02PM -0700, H. Peter Anvin wrote:

> > I2C or similar busses can be a particularly annoying if they contain
> > essential configuration information such as memory size which is needed
> > long before anything else.  So for far a common solution is that platforms
> > are carrying a private (aka redundant, ugly) early-i2c system that's just
> > about sufficient for this purpose.
> 
> For what it's worth, this is true for pretty much ALL systems with
> removable memory modules, since Serial Presence Detect (SPD) is
> electrically equivalent to I2C.
> 
> However, on most systems, even embedded, bringing up memory falls on
> firmware (sometimes in the form of a boot loader) so Linux rarely sees it.

There are embedded systems were the firmware does not provide a usuable
memory map or where that is plain broken.  Or Linux with some extra init
code serves as the firmware.  Often there is a single serial EEPROM for
the entire system.  If there is an atrocity that can save a penny it will
be commited at least in the embedded world.

  Ralf

Re: [Ksummit-2009-discuss] Representing Embedded Architectures at the Kernel Summit

[H. Peter Anvin]

Ralf Baechle wrote:
>>
>> However, on most systems, even embedded, bringing up memory falls on
>> firmware (sometimes in the form of a boot loader) so Linux rarely sees it.
> 
> There are embedded systems were the firmware does not provide a usuable
> memory map or where that is plain broken.  Or Linux with some extra init
> code serves as the firmware.  Often there is a single serial EEPROM for
> the entire system.  If there is an atrocity that can save a penny it will
> be commited at least in the embedded world.
> 

"Rarely" is certainly not "never".  Quite on the contrary.  Also, I
think you can remove "that can save a penny" from your last sentence...

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 16, 2009 at 2:06 AM, Mike Frysinger<vapier.adi@gmail.com> wrote:
> On Tue, Jun 16, 2009 at 02:42, Mike Rapoport wrote:
>> James Bottomley wrote:
>> Another issue that affects embedded architectures is drivers initialization
>> order. There are a lot of cases when you need the drivers to be initialized in
>> particular order, and current initcalls scheme does not allow fine grained
>> control for it.
>
> example: device configuration information stored in i2c eeprom (i.e.
> dimensions of attached framebuffer), but i2c is not available when
> framebuffer layer is setup.  framebuffer driver has to be built as a
> module and loaded by userspace, or i2c information is read by
> bootloader and passed down to the kernel.

I experimented a bit with having some infrastructure for waiting for
another device to get either registered as part of the phylib stuff I
was doing.  Here's the patchwork link to the discussion:

http://patchwork.ozlabs.org/patch/24152/

I never actually pushed through and finished it because it turned out
to be a non-issue for Ethernet devices in the end.  However, I can see
the value.  With this approach, a driver can use a
bus_register_notifier() variant without caring about the device
registration order, and the drivers notifier callback will get called
at the appropriate time.

In your example case I could see the framebuffer driver deferring the
final part of its initialization until the needed i2c device shows up.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Jamie Lokier]

Grant Likely wrote:
> http://patchwork.ozlabs.org/patch/24152/
> 
> I never actually pushed through and finished it because it turned out
> to be a non-issue for Ethernet devices in the end.  However, I can see
> the value.  With this approach, a driver can use a
> bus_register_notifier() variant without caring about the device
> registration order, and the drivers notifier callback will get called
> at the appropriate time.
> 
> In your example case I could see the framebuffer driver deferring the
> final part of its initialization until the needed i2c device shows up.

It makes drivers more complicated and larger than necessary, though,
having to add more functions and register more callbacks.

Something which lets you specify a dependency in a one-line
MODULE_INIT_PREREQS() macro would be much nicer.

I can see the callbacks being useful, and necessary, for drivers which
need to interleave their initialisation.  E.g. need to initialise an
external framebuffer chip before you can use it's I2C GPIOs, but then
need to initialise an I2C driver before you can read the framebuffer
parameters to finish setting up the framebuffer.

-- Jamie

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 16, 2009 at 12:18 PM, Jamie Lokier<jamie@shareable.org> wrote:
> Grant Likely wrote:
>> http://patchwork.ozlabs.org/patch/24152/
>>
>> I never actually pushed through and finished it because it turned out
>> to be a non-issue for Ethernet devices in the end.  However, I can see
>> the value.  With this approach, a driver can use a
>> bus_register_notifier() variant without caring about the device
>> registration order, and the drivers notifier callback will get called
>> at the appropriate time.
>>
>> In your example case I could see the framebuffer driver deferring the
>> final part of its initialization until the needed i2c device shows up.
>
> It makes drivers more complicated and larger than necessary, though,
> having to add more functions and register more callbacks.
>
> Something which lets you specify a dependency in a one-line
> MODULE_INIT_PREREQS() macro would be much nicer.

That would work for some cases, but a lot of cases the problem is not
module init order, but rather driver the probe order.  ie. In the
Ethernet case I was working on the mac cannot be opened before the
phy_device is registered.  Or another example is GPIOs.  An GPIO
driven SPI or MDIO bus cannot be probed before all the devices
providing the GPIOs are probed (right now GPIOs are 'special' and
probed early, but this is ugly and there is no reason it couldn't be
handled within the Linux driver model.

I could see some form of .probe_prereqs member of a device driver, but
what form should it take?  Callback?  Data structure?  The field of
prereqs (or rather figuring out the prereqs) is so large that I don't
feel confident trying to define something.  A callback is probably the
most flexible if it could be called after each probe or registration
event on other devices.  The idea being that if the prereq call fails,
then it will be tried again later.  Or, thinking further, perhaps
instead of a .probe_prereq, just have .probe returns -ETRYAGAIN and
make the driver core put the device on the end of the queue of devices
to get probed.  This could get ugly in the driver core though.

Needs more thought....

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Jamie Lokier]

Grant Likely wrote:
> On Tue, Jun 16, 2009 at 12:18 PM, Jamie Lokier<jamie@shareable.org> wrote:
> > Something which lets you specify a dependency in a one-line
> > MODULE_INIT_PREREQS() macro would be much nicer.
> 
> That would work for some cases, but a lot of cases the problem is not
> module init order, but rather driver the probe order.  ie. In the
> Ethernet case I was working on the mac cannot be opened before the
> phy_device is registered.  Or another example is GPIOs.  An GPIO
> driven SPI or MDIO bus cannot be probed before all the devices
> providing the GPIOs are probed (right now GPIOs are 'special' and
> probed early, but this is ugly and there is no reason it couldn't be
> handled within the Linux driver model.

Both of those cases look like a simple module init order problem.

I'm not seeing how it's a probe order problem.  Even if you can probe
PHYs independently first, how are they going to be bound to the
ethernet MACs, other than by dodgy defaults?

In any of your examples, is the "modprobe" symbol dependency order
insufficient, when they are external modules?

If the problem is simply that "modprobe" can calculate dependencies
but linked-in modules don't, maybe the solution is to use the symbolic
dependencies to calculate a linked-in driver initialisation order.

If it's a probe order problem, where there aren't symbolic
dependencies, then MODULE_PROVIDE("gpio") and MODULE_REQUIRE("gpio")
or something like that might handle those cases, except for tricky
ones like a GPIO-driven I2C bus which controls a GPIO chip.

-- Jamie

Re: Representing Embedded Architectures at the Kernel Summit

[Marcel Holtmann]

Hi Jamie,

> > > Something which lets you specify a dependency in a one-line
> > > MODULE_INIT_PREREQS() macro would be much nicer.
> > 
> > That would work for some cases, but a lot of cases the problem is not
> > module init order, but rather driver the probe order.  ie. In the
> > Ethernet case I was working on the mac cannot be opened before the
> > phy_device is registered.  Or another example is GPIOs.  An GPIO
> > driven SPI or MDIO bus cannot be probed before all the devices
> > providing the GPIOs are probed (right now GPIOs are 'special' and
> > probed early, but this is ugly and there is no reason it couldn't be
> > handled within the Linux driver model.
> 
> Both of those cases look like a simple module init order problem.
> 
> I'm not seeing how it's a probe order problem.  Even if you can probe
> PHYs independently first, how are they going to be bound to the
> ethernet MACs, other than by dodgy defaults?
> 
> In any of your examples, is the "modprobe" symbol dependency order
> insufficient, when they are external modules?
> 
> If the problem is simply that "modprobe" can calculate dependencies
> but linked-in modules don't, maybe the solution is to use the symbolic
> dependencies to calculate a linked-in driver initialisation order.
> 
> If it's a probe order problem, where there aren't symbolic
> dependencies, then MODULE_PROVIDE("gpio") and MODULE_REQUIRE("gpio")
> or something like that might handle those cases, except for tricky
> ones like a GPIO-driven I2C bus which controls a GPIO chip.

for the loading order so far I have always added a dummy symbol to
ensure that loading of modules are in the right order. Seems to be a
good enough approach.

Regards

Marcel

Re: Representing Embedded Architectures at the Kernel Summit

[Grant Likely]

On Tue, Jun 16, 2009 at 2:07 PM, Jamie Lokier<jamie@shareable.org> wrote:
> Grant Likely wrote:
>> On Tue, Jun 16, 2009 at 12:18 PM, Jamie Lokier<jamie@shareable.org> wrote:
>> > Something which lets you specify a dependency in a one-line
>> > MODULE_INIT_PREREQS() macro would be much nicer.
>>
>> That would work for some cases, but a lot of cases the problem is not
>> module init order, but rather driver the probe order.  ie. In the
>> Ethernet case I was working on the mac cannot be opened before the
>> phy_device is registered.  Or another example is GPIOs.  An GPIO
>> driven SPI or MDIO bus cannot be probed before all the devices
>> providing the GPIOs are probed (right now GPIOs are 'special' and
>> probed early, but this is ugly and there is no reason it couldn't be
>> handled within the Linux driver model.
>
> Both of those cases look like a simple module init order problem.
>
> I'm not seeing how it's a probe order problem.  Even if you can probe
> PHYs independently first, how are they going to be bound to the
> ethernet MACs, other than by dodgy defaults?
>
> In any of your examples, is the "modprobe" symbol dependency order
> insufficient, when they are external modules?

In both cases the driver is waiting for a specific instance, but has
no idea which driver will provide that service.

For the Ethernet example:  Both the MAC driver and the MDIO bus driver
are registered independently (say via either the platform bus).  The
MDIO bus driver creates a new mdio_bus instance and registers a
phy_device for each child on the bus.  The pdata for the MAC driver
only contains the address of the phy, but it does not know which
driver handles the MDIO bus and it does not know what driver will
handle the phy.  All the MAC cares about dereferencing the phy address
to get a pointer to the phy_device structure which only works after
the MDIO bus is probed and the correct phy_device is registered..
Module dependencies won't help here because the MAC cannot know what
module to depend against.  Plus, in some cases, the MAC will be used
without a PHY, in which case it cannot depend on any phy modules.

For the GPIO example, same thing.  Something like the i2c-gpio driver
doesn't know anything about which device provides the GPIO signals.
It is only handled GPIO numbers for each signal and it cannot be
probed before all the required GPIOs are registered with the system.
Again module dependencies don't work because the driver doesn't know
which GPIO driver it will get bound against at run time.

> If the problem is simply that "modprobe" can calculate dependencies
> but linked-in modules don't, maybe the solution is to use the symbolic
> dependencies to calculate a linked-in driver initialisation order.
>
> If it's a probe order problem, where there aren't symbolic
> dependencies, then MODULE_PROVIDE("gpio") and MODULE_REQUIRE("gpio")
> or something like that might handle those cases, except for tricky
> ones like a GPIO-driven I2C bus which controls a GPIO chip.

Ugh.... it feels too coarse grained to me.  It solves some of the
problems, but still requires playing dirty tricks for the tricky cases
which scuttles the whole lot.  ie. your i2c GPIO chip example.
Another driver wired to one of those 2nd level GPIOs and using
MODULE_REQUIRE("gpio") would immediately be defeated and would need
some other method to indicate the dependency.  Plus there is the
question when stuff is loaded as modules as to how does the kernel
know when MODULE_PROVIDE("gpio") is complete?  What if I load the gpio
driver after the gpio-i2c driver?  I'd much rather define the
dependency at the resource level, not module level because that is
what the driver is actually needing.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Representing Embedded Architectures at the Kernel Summit

[Paul Mundt]

On Tue, Jun 16, 2009 at 09:42:46AM +0300, Mike Rapoport wrote:
> James Bottomley wrote:
> > We've got to the point where there are simply too many embedded
> > architectures to invite all the arch maintainers to the kernel summit.
> > So, this year, we thought we'd do embedded via topic driven invitations
> > instead.  So what we're looking for is a proposal to discuss the issues
> > most affecting embedded architectures, or preview any features affecting
> > the main kernel which embedded architectures might need ... or any other
> > topics from embedded architectures which might need discussion or
> > debate.
> 
> Another issue that affects embedded architectures is drivers initialization
> order. There are a lot of cases when you need the drivers to be initialized in
> particular order, and current initcalls scheme does not allow fine grained
> control for it.
> 
Look at the early platform device abstraction, this allows specific
fine-grained control over when certain drivers are initialized, well
before the driver core is available. On SH this is how we deal with 
our system timers as clockevents/clocksources while just using regular
platform devices, and having no other abstraction. You can read more in
Documentation/driver-model/platform.txt.  For an example,  you can grep
for earlytimer in arch/sh as well as in drivers/clocksource.

Re: Representing Embedded Architectures at the Kernel Summit

[Kumar Gala]

One topic that was partially touched on was dealing with various  
memories on embedded systems.  We have several sram based allocators  
in the kernel for various different arch's:

- Blackfin sram allocator arch/blackfin/mm/sram-alloc.c
- Lite5200(b) sram allocator arch/powerpc/sysdev/bestcomm/sram.c
- AVR32 sram allocator arch/avr32/mach-at32ap/at32ap700x.c and arch/ 
avr32/mach-at32ap/include/mach/sram.h
- Potential davinci sram allocator

There maybe others.

- k

Re: Representing Embedded Architectures at the Kernel Summit

[Paul Mundt]

On Wed, Jun 17, 2009 at 09:31:48AM -0500, Kumar Gala wrote:
> One topic that was partially touched on was dealing with various  
> memories on embedded systems.  We have several sram based allocators  
> in the kernel for various different arch's:
> 
> - Blackfin sram allocator arch/blackfin/mm/sram-alloc.c
> - Lite5200(b) sram allocator arch/powerpc/sysdev/bestcomm/sram.c
> - AVR32 sram allocator arch/avr32/mach-at32ap/at32ap700x.c and arch/ 
> avr32/mach-at32ap/include/mach/sram.h
> - Potential davinci sram allocator
> 
> There maybe others.
> 
SH does this through NUMA on SRAM blocks that are anywhere from 128kB to
64MB. Some of our SMP configurations have upwards of a dozen of these
blocks.

Re: Representing Embedded Architectures at the Kernel Summit

[Kumar Gala]

On Jun 17, 2009, at 9:51 PM, Paul Mundt wrote:

> On Wed, Jun 17, 2009 at 09:31:48AM -0500, Kumar Gala wrote:
>> One topic that was partially touched on was dealing with various
>> memories on embedded systems.  We have several sram based allocators
>> in the kernel for various different arch's:
>>
>> - Blackfin sram allocator arch/blackfin/mm/sram-alloc.c
>> - Lite5200(b) sram allocator arch/powerpc/sysdev/bestcomm/sram.c
>> - AVR32 sram allocator arch/avr32/mach-at32ap/at32ap700x.c and arch/
>> avr32/mach-at32ap/include/mach/sram.h
>> - Potential davinci sram allocator
>>
>> There maybe others.
>>
> SH does this through NUMA on SRAM blocks that are anywhere from  
> 128kB to
> 64MB. Some of our SMP configurations have upwards of a dozen of these
> blocks.

Do you really have that much on chip memory?

- k

Re: Representing Embedded Architectures at the Kernel Summit

[Paul Mundt]

On Thu, Jun 18, 2009 at 09:59:20PM -0500, Kumar Gala wrote:
> 
> On Jun 17, 2009, at 9:51 PM, Paul Mundt wrote:
> 
> >On Wed, Jun 17, 2009 at 09:31:48AM -0500, Kumar Gala wrote:
> >>One topic that was partially touched on was dealing with various
> >>memories on embedded systems.  We have several sram based allocators
> >>in the kernel for various different arch's:
> >>
> >>- Blackfin sram allocator arch/blackfin/mm/sram-alloc.c
> >>- Lite5200(b) sram allocator arch/powerpc/sysdev/bestcomm/sram.c
> >>- AVR32 sram allocator arch/avr32/mach-at32ap/at32ap700x.c and arch/
> >>avr32/mach-at32ap/include/mach/sram.h
> >>- Potential davinci sram allocator
> >>
> >>There maybe others.
> >>
> >SH does this through NUMA on SRAM blocks that are anywhere from  
> >128kB to
> >64MB. Some of our SMP configurations have upwards of a dozen of these
> >blocks.
> 
> Do you really have that much on chip memory?
> 
Is this really a serious question? If we didn't, I would not have
mentioned it.

Re: Representing Embedded Architectures at the Kernel Summit

[Kumar Gala]

On Jun 18, 2009, at 10:00 PM, Paul Mundt wrote:

> On Thu, Jun 18, 2009 at 09:59:20PM -0500, Kumar Gala wrote:
>>
>> On Jun 17, 2009, at 9:51 PM, Paul Mundt wrote:
>>
>>> On Wed, Jun 17, 2009 at 09:31:48AM -0500, Kumar Gala wrote:
>>>> One topic that was partially touched on was dealing with various
>>>> memories on embedded systems.  We have several sram based  
>>>> allocators
>>>> in the kernel for various different arch's:
>>>>
>>>> - Blackfin sram allocator arch/blackfin/mm/sram-alloc.c
>>>> - Lite5200(b) sram allocator arch/powerpc/sysdev/bestcomm/sram.c
>>>> - AVR32 sram allocator arch/avr32/mach-at32ap/at32ap700x.c and  
>>>> arch/
>>>> avr32/mach-at32ap/include/mach/sram.h
>>>> - Potential davinci sram allocator
>>>>
>>>> There maybe others.
>>>>
>>> SH does this through NUMA on SRAM blocks that are anywhere from
>>> 128kB to
>>> 64MB. Some of our SMP configurations have upwards of a dozen of  
>>> these
>>> blocks.
>>
>> Do you really have that much on chip memory?
>>
> Is this really a serious question? If we didn't, I would not have
> mentioned it.

It was.  I'm not familiar with SH hardware so wasn't sure if there was  
some other application you had for the multiple regions.

- k