kernel entry for thumb2-only cpus

kernel entry for thumb2-only cpus

[Uwe Kleine-König]

Hello,

in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have

	.arm
 THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
 THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
 THUMB( .thumb                  )       @ switch to Thumb now.
 THUMB(1:                       )

as first instructions at the entry point. This is a problem for
thumb2-only cpus (e.g. Cortex-M3).

Up to now I commented out the first three lines for the Cortex-M3 port.
What should we do about that. There are two possibilities I see:

 a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
    first three lines out if it is given.
 b) expect the bootloader to directly jump to the label 1.

The downside of a) is that a boot loader on a cpu that is capable to
execute the tradtional instructions would need to detect if the switch
to thumb is there or not and react accordingly. (In fact it needs to
distringuish three cases:

 - traditional kernel
 - thumb2 kernel with ARM entry
 - thumb2 kernel without ARM entry

.) So I think b) is the more sensible option in the long run.

What do you think?

If so I'd expand the comment with this expectation.

Best regards
Uwe

-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

kernel entry for thumb2-only cpus

[Will Deacon]

On Fri, Aug 03, 2012 at 10:04:52AM +0100, Uwe Kleine-K?nig wrote:
> Hello,

Hi Uwe,

> in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have
> 
> 	.arm
>  THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
>  THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
>  THUMB( .thumb                  )       @ switch to Thumb now.
>  THUMB(1:                       )
> 
> as first instructions at the entry point. This is a problem for
> thumb2-only cpus (e.g. Cortex-M3).

Yup, Documentation/arm/Booting mentions this too.

> Up to now I commented out the first three lines for the Cortex-M3 port.
> What should we do about that. There are two possibilities I see:
> 
>  a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
>     first three lines out if it is given.
>  b) expect the bootloader to directly jump to the label 1.
> 
> The downside of a) is that a boot loader on a cpu that is capable to
> execute the tradtional instructions would need to detect if the switch
> to thumb is there or not and react accordingly. (In fact it needs to
> distringuish three cases:
> 
>  - traditional kernel
>  - thumb2 kernel with ARM entry
>  - thumb2 kernel without ARM entry
> 
> .) So I think b) is the more sensible option in the long run.
> 
> What do you think?

How about something like:


diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 835898e..9f07be2 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -86,6 +86,7 @@
        __HEAD
 ENTRY(stext)
 
+ THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
  THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
  THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
  THUMB(        .thumb                  )       @ switch to Thumb now.


[thanks to Leif Lindholm for doing this in one instruction]

Will

kernel entry for thumb2-only cpus

[Uwe Kleine-König]

On Fri, Aug 03, 2012 at 12:45:32PM +0100, Will Deacon wrote:
> On Fri, Aug 03, 2012 at 10:04:52AM +0100, Uwe Kleine-K?nig wrote:
> > Hello,
> 
> Hi Uwe,
> 
> > in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have
> > 
> > 	.arm
> >  THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> >  THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
> >  THUMB( .thumb                  )       @ switch to Thumb now.
> >  THUMB(1:                       )
> > 
> > as first instructions at the entry point. This is a problem for
> > thumb2-only cpus (e.g. Cortex-M3).
> 
> Yup, Documentation/arm/Booting mentions this too.
> 
> > Up to now I commented out the first three lines for the Cortex-M3 port.
> > What should we do about that. There are two possibilities I see:
> > 
> >  a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
> >     first three lines out if it is given.
> >  b) expect the bootloader to directly jump to the label 1.
> > 
> > The downside of a) is that a boot loader on a cpu that is capable to
> > execute the tradtional instructions would need to detect if the switch
> > to thumb is there or not and react accordingly. (In fact it needs to
> > distringuish three cases:
> > 
> >  - traditional kernel
> >  - thumb2 kernel with ARM entry
> >  - thumb2 kernel without ARM entry
> > 
> > .) So I think b) is the more sensible option in the long run.
> > 
> > What do you think?
> 
> How about something like:
> 
> 
> diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> index 835898e..9f07be2 100644
> --- a/arch/arm/kernel/head.S
> +++ b/arch/arm/kernel/head.S
> @@ -86,6 +86,7 @@
>         __HEAD
>  ENTRY(stext)
>  
> + THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
>   THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
>   THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
>   THUMB(        .thumb                  )       @ switch to Thumb now.
Great idea, but it doesn't work as suggested. My compiler already barfs
on the .arm above:

	arch/arm/kernel/head-nommu.S:36: Error: selected processor does not support ARM opcodes
	arch/arm/kernel/head-nommu.S:42: Error: attempt to use an ARM instruction on a Thumb-only processor -- `adr r9,1f+1'
	arch/arm/kernel/head-nommu.S:43: Error: attempt to use an ARM instruction on a Thumb-only processor -- `bx r9'

but something like that should be doable.

Best regards
Uwe


-- 
Pengutronix e.K.                           | Uwe Kleine-K?nig            |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |

kernel entry for thumb2-only cpus

[Will Deacon]

On Sun, Aug 05, 2012 at 10:55:28AM +0100, Uwe Kleine-K?nig wrote:
> On Fri, Aug 03, 2012 at 12:45:32PM +0100, Will Deacon wrote:
> > How about something like:
> > 
> > 
> > diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> > index 835898e..9f07be2 100644
> > --- a/arch/arm/kernel/head.S
> > +++ b/arch/arm/kernel/head.S
> > @@ -86,6 +86,7 @@
> >         __HEAD
> >  ENTRY(stext)
> >  
> > + THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
> >   THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> >   THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
> >   THUMB(        .thumb                  )       @ switch to Thumb now.
> Great idea, but it doesn't work as suggested. My compiler already barfs
> on the .arm above:
> 
> 	arch/arm/kernel/head-nommu.S:36: Error: selected processor does not support ARM opcodes
> 	arch/arm/kernel/head-nommu.S:42: Error: attempt to use an ARM instruction on a Thumb-only processor -- `adr r9,1f+1'
> 	arch/arm/kernel/head-nommu.S:43: Error: attempt to use an ARM instruction on a Thumb-only processor -- `bx r9'

Aaand our good friend gas bites us again. Out of interest, what happens 
if you pass -march=all instead of whatever you currently pass (armv7m or
something I guess?). We hit a bug the other day assembling the decompressor
and I'd be interested to know what the tools do with your code.

> but something like that should be doable.

Maybe we should just .inst the two arm instructions as well and lose the
.arm directive. Given that the size of those instructions is now encoded
in the first instruction (for the Thumb-2 branch), it would also make it
clear that you can't just add extra arm instructions there without
changing the branch.

Will

kernel entry for thumb2-only cpus

[Nicolas Pitre]

On Sun, 5 Aug 2012, Uwe Kleine-K?nig wrote:

> On Fri, Aug 03, 2012 at 12:45:32PM +0100, Will Deacon wrote:
> > On Fri, Aug 03, 2012 at 10:04:52AM +0100, Uwe Kleine-K?nig wrote:
> > > Hello,
> > 
> > Hi Uwe,
> > 
> > > in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have
> > > 
> > > 	.arm
> > >  THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> > >  THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
> > >  THUMB( .thumb                  )       @ switch to Thumb now.
> > >  THUMB(1:                       )
> > > 
> > > as first instructions at the entry point. This is a problem for
> > > thumb2-only cpus (e.g. Cortex-M3).
> > 
> > Yup, Documentation/arm/Booting mentions this too.
> > 
> > > Up to now I commented out the first three lines for the Cortex-M3 port.
> > > What should we do about that. There are two possibilities I see:
> > > 
> > >  a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
> > >     first three lines out if it is given.
> > >  b) expect the bootloader to directly jump to the label 1.
> > > 
> > > The downside of a) is that a boot loader on a cpu that is capable to
> > > execute the tradtional instructions would need to detect if the switch
> > > to thumb is there or not and react accordingly. (In fact it needs to
> > > distringuish three cases:
> > > 
> > >  - traditional kernel
> > >  - thumb2 kernel with ARM entry
> > >  - thumb2 kernel without ARM entry
> > > 
> > > .) So I think b) is the more sensible option in the long run.
> > > 
> > > What do you think?
> > 
> > How about something like:
> > 
> > 
> > diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> > index 835898e..9f07be2 100644
> > --- a/arch/arm/kernel/head.S
> > +++ b/arch/arm/kernel/head.S
> > @@ -86,6 +86,7 @@
> >         __HEAD
> >  ENTRY(stext)
> >  
> > + THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
> >   THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> >   THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
> >   THUMB(        .thumb                  )       @ switch to Thumb now.
> Great idea, but it doesn't work as suggested. My compiler already barfs
> on the .arm above:

Let's stop splitting hairs.

A Cortex-M kernel simply won't run on anything else.  It certainly won't 
run on ARM mode capable processors.  So in this case just make the 
kernel entry point Thumb mode only by #ifdef'ing out the .arm part.

This is good _only_ for CPUs without any ARM mode capability though.


Nicolas

kernel entry for thumb2-only cpus

[Matt Sealey]

On Sun, Aug 5, 2012 at 11:32 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Sun, 5 Aug 2012, Uwe Kleine-K?nig wrote:
>
>> On Fri, Aug 03, 2012 at 12:45:32PM +0100, Will Deacon wrote:
>> > On Fri, Aug 03, 2012 at 10:04:52AM +0100, Uwe Kleine-K?nig wrote:
>> > > Hello,
>> >
>> > Hi Uwe,
>> >
>> > > in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have
>> > >
>> > >   .arm
>> > >  THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
>> > >  THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
>> > >  THUMB( .thumb                  )       @ switch to Thumb now.
>> > >  THUMB(1:                       )
>> > >
>> > > as first instructions at the entry point. This is a problem for
>> > > thumb2-only cpus (e.g. Cortex-M3).
>> >
>> > Yup, Documentation/arm/Booting mentions this too.
>> >
>> > > Up to now I commented out the first three lines for the Cortex-M3 port.
>> > > What should we do about that. There are two possibilities I see:
>> > >
>> > >  a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
>> > >     first three lines out if it is given.
>> > >  b) expect the bootloader to directly jump to the label 1.
>> > >
>> > > The downside of a) is that a boot loader on a cpu that is capable to
>> > > execute the tradtional instructions would need to detect if the switch
>> > > to thumb is there or not and react accordingly. (In fact it needs to
>> > > distringuish three cases:
>> > >
>> > >  - traditional kernel
>> > >  - thumb2 kernel with ARM entry
>> > >  - thumb2 kernel without ARM entry
>> > >
>> > > .) So I think b) is the more sensible option in the long run.
>> > >
>> > > What do you think?
>> >
>> > How about something like:
>> >
>> >
>> > diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
>> > index 835898e..9f07be2 100644
>> > --- a/arch/arm/kernel/head.S
>> > +++ b/arch/arm/kernel/head.S
>> > @@ -86,6 +86,7 @@
>> >         __HEAD
>> >  ENTRY(stext)
>> >
>> > + THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
>> >   THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
>> >   THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
>> >   THUMB(        .thumb                  )       @ switch to Thumb now.
>> Great idea, but it doesn't work as suggested. My compiler already barfs
>> on the .arm above:
>
> Let's stop splitting hairs.
>
> A Cortex-M kernel simply won't run on anything else.  It certainly won't
> run on ARM mode capable processors.  So in this case just make the
> kernel entry point Thumb mode only by #ifdef'ing out the .arm part.
>
> This is good _only_ for CPUs without any ARM mode capability though.

Wouldn't it be nice to have a solution where you can build a Thumb2
bootloader (like U-Boot which can do this) for an ARM-with-Thumb2
processor and not needlessly enter/exit Thumb2 mode while entering the
kernel? It's not something that could be automated but it might make
for something a little more deterministic in terms of running "the
same code" on two only slightly different architectures (after all why
else would you want to run Linux on a Cortex-M? Most microcontroller
development shuns Linux :)

Could we have an option that basically is subordinate to
CONFIG_THUMB2_KERNEL such as CONFIG_THUMB2_ENTRYPOINT (depends on
CONFIG_THUMB2) with a huge warning in the enable that this won't boot
from U-Boot that's not compiled to support it (and there is a patch
for this somewhere that someone has to write to support it) and/or
your bootloader absolutely MUST be in Thumb2 mode before it jumps to
the entrypoint to avoid an exception?

Maybe we could implement a shiny new zImage magic value, I'm not sure
if 0x016f2818 might have some sentimental value, but there's nothing
to say that 0x016f2819 wouldn't also be valid and indicate that it's
an ARM zImage with a Thumb2 entrypoint?

What kind of images are we expecting to load on a Cortex-M anyway? I
would assume more common than a zImage would be some kind of special
compressed image somehow since flash would be at a premium... you
don't want to bulk your kernel up with 64kb of U-Boot SPL or so, or
even anything smaller than that. In this case, since surely Cortex-M
boots up in a more amenable state to loading kernels from flash (or
flexram or flexnvm or whatever the other manufacturers call it, and
all the docs say you can run pretty much C code from first instruction
instead of the assembler headers usually setting up exception vectors)
by just decompressing themselves automatically anyway. There, you
would not need the ARM code at all.

In the case where you'd rather run a bootloader and run a kernel from
removable storage or over USB or ethernet, you're stuck with an
external bootloader like U-Boot, and the compilation of both is going
to be controlled, one would hope; in this case, you'd configure your
loader for Thumb and your kernel for Thumb.

There's possibly some use in this, not a lot of use, but at least you
could guarantee your backtrace at early boot would be the same on M4
and A9 if it never swapped in and out of ARM mode just to execute
Thumb code exclusively from U-Boot through to Linux. Maybe that counts
for some people.. I personally like developing code on faster
processors before putting them on the slower ones, and the fact the
ISA is tantamount to identical is one of the great reasons to use the
M4..

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Russell King - ARM Linux]

On Mon, Aug 06, 2012 at 01:40:26PM -0500, Matt Sealey wrote:
> Maybe we could implement a shiny new zImage magic value, I'm not sure
> if 0x016f2818 might have some sentimental value, but there's nothing
> to say that 0x016f2819 wouldn't also be valid and indicate that it's
> an ARM zImage with a Thumb2 entrypoint?

No.  That magic value is a bloody pain in the backside from my point
of view.  I've had soo many problems with it with boot loaders its
untrue.

For example, the Skiff (SA110 platform produced by Compaq) using
bootldr.  It detects the value in the kernel image after loading
the kernel at 0x8000 into RAM, and then proceeds to move the entire
kernel image to RAM location 0, and then calls it at 0x8000 !

Naturally, that action corrupts the ATAG list etc.

To work around that, I've ended up doing some weird and wonderful
tricks like pre-pending 32K of zeros, and copying the first 128
bytes of the kernel image to the start of the image file to fool
the boot loader.  Still results in a corrupted ATAG list but it is
bootable.

There's other boot loaders around too which change their behaviour
in weird and wonderful ways depending on magic values in the loaded
image file.

What this all means is that stuff like this is at the mercy of the
boot loader folk, and when problems happen, it's hard to work around
them, because you have no way to tell the boot loader to do something
different.

Instead, lets not continue this broken idea.  Instead, let's be able
to _tell_ the boot loader what we want it to do, rather than have it
magically decipher what _it_ thinks might be a good idea to do.

kernel entry for thumb2-only cpus

[Matt Sealey]

On Mon, Aug 6, 2012 at 2:08 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Aug 06, 2012 at 01:40:26PM -0500, Matt Sealey wrote:
>> Maybe we could implement a shiny new zImage magic value, I'm not sure
>> if 0x016f2818 might have some sentimental value, but there's nothing
>> to say that 0x016f2819 wouldn't also be valid and indicate that it's
>> an ARM zImage with a Thumb2 entrypoint?
>
> No.  That magic value is a bloody pain in the backside from my point
> of view.  I've had soo many problems with it with boot loaders its
> untrue.

[snip]

> Instead, lets not continue this broken idea.  Instead, let's be able
> to _tell_ the boot loader what we want it to do, rather than have it
> magically decipher what _it_ thinks might be a good idea to do.

Okay, so how about we just append an ELF header to the front of the
image so the entry point, and the architecture, instruction
set, ABI etc. is well defined in an industry standard manner?

e_entry would then have the LSB set which indicates the entry is
Thumb.. or not :)

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Russell King - ARM Linux]

On Mon, Aug 06, 2012 at 02:30:48PM -0500, Matt Sealey wrote:
> On Mon, Aug 6, 2012 at 2:08 PM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
> > On Mon, Aug 06, 2012 at 01:40:26PM -0500, Matt Sealey wrote:
> >> Maybe we could implement a shiny new zImage magic value, I'm not sure
> >> if 0x016f2818 might have some sentimental value, but there's nothing
> >> to say that 0x016f2819 wouldn't also be valid and indicate that it's
> >> an ARM zImage with a Thumb2 entrypoint?
> >
> > No.  That magic value is a bloody pain in the backside from my point
> > of view.  I've had soo many problems with it with boot loaders its
> > untrue.
> 
> [snip]
> 
> > Instead, lets not continue this broken idea.  Instead, let's be able
> > to _tell_ the boot loader what we want it to do, rather than have it
> > magically decipher what _it_ thinks might be a good idea to do.
> 
> Okay, so how about we just append an ELF header to the front of the
> image so the entry point, and the architecture, instruction
> set, ABI etc. is well defined in an industry standard manner?
> 
> e_entry would then have the LSB set which indicates the entry is
> Thumb.. or not :)

No.  Take a moment to think about what you're saying.  Are you really
saying no to the single kernel image project?

Because anything that involves wrapping the kernel up in an "industry
standard manner" like an ELF header places fixed addresses (like the
entry point) into the kernel, and then we're back to the same problem
that every SoC vendor places their RAM at different places in the
system memory map.

No.  Can't do that.  Sorry.

kernel entry for thumb2-only cpus

[Matt Sealey]

On Mon, Aug 6, 2012 at 2:40 PM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Mon, Aug 06, 2012 at 02:30:48PM -0500, Matt Sealey wrote:
>> On Mon, Aug 6, 2012 at 2:08 PM, Russell King - ARM Linux
>> <linux@arm.linux.org.uk> wrote:
>> > On Mon, Aug 06, 2012 at 01:40:26PM -0500, Matt Sealey wrote:
>> >> Maybe we could implement a shiny new zImage magic value, I'm not sure
>> >> if 0x016f2818 might have some sentimental value, but there's nothing
>> >> to say that 0x016f2819 wouldn't also be valid and indicate that it's
>> >> an ARM zImage with a Thumb2 entrypoint?
>> >
>> > No.  That magic value is a bloody pain in the backside from my point
>> > of view.  I've had soo many problems with it with boot loaders its
>> > untrue.
>>
>> [snip]
>>
>> > Instead, lets not continue this broken idea.  Instead, let's be able
>> > to _tell_ the boot loader what we want it to do, rather than have it
>> > magically decipher what _it_ thinks might be a good idea to do.
>>
>> Okay, so how about we just append an ELF header to the front of the
>> image so the entry point, and the architecture, instruction
>> set, ABI etc. is well defined in an industry standard manner?
>>
>> e_entry would then have the LSB set which indicates the entry is
>> Thumb.. or not :)
>
> No.  Take a moment to think about what you're saying.  Are you really
> saying no to the single kernel image project?
>
> Because anything that involves wrapping the kernel up in an "industry
> standard manner" like an ELF header places fixed addresses (like the
> entry point) into the kernel, and then we're back to the same problem
> that every SoC vendor places their RAM at different places in the
> system memory map.
>
> No.  Can't do that.  Sorry.

Right now there's already a fixed entry point - ARM kernels are fixed
entry point of relative 0 to the start of the loaded image. Okay, so
we can't format a totally valid ELF header, but in the event that we
want to encode all this information yet again, why reinvent the wheel?

Just reserve your bytes or so at the front of the image for head.S,
and at the correct point (right where the zImage magic would be) sits
an ELF magic and the rest of the ELF header. I realize you'd need to
write another rudimentary ELF parser for U-Boot and one for very early
init in the kernel to get the jump right, therefore reinventing the
wheel in some respects, but I'm curious how anyone could do any better
with another custom image header format. It's somewhat this way anyway
(to my last recollection vmlinux is ELF so you can get gdb to load it)
and the only better thing I can think of is to use PECOFF, because
then it'd automatically be UEFI compatible (and we can all stop using
grub-efi and directly boot a kernel).

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Nicolas Pitre]

On Mon, 6 Aug 2012, Matt Sealey wrote:

> Right now there's already a fixed entry point - ARM kernels are fixed
> entry point of relative 0 to the start of the loaded image. Okay, so
> we can't format a totally valid ELF header, but in the event that we
> want to encode all this information yet again, why reinvent the wheel?

That's the point.  We do _not_ want to encode anything.

By convention, if your CPU is ARM mode capable, then it must call the 
kernel in ARM mode, period.  Can't be simpler than that, right?  The 
bootloader shouldn't care if the kernel is itself ARM or Thumb, and the 
kernel shouldn't care if the bootloader is ARM or Thumb.

If you have a real problem with this then let's hear it so it can be 
addressed.


Nicolas

kernel entry for thumb2-only cpus

[Matt Sealey]

On Mon, Aug 6, 2012 at 5:36 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Mon, 6 Aug 2012, Matt Sealey wrote:
>
>> Right now there's already a fixed entry point - ARM kernels are fixed
>> entry point of relative 0 to the start of the loaded image. Okay, so
>> we can't format a totally valid ELF header, but in the event that we
>> want to encode all this information yet again, why reinvent the wheel?
>
> That's the point.  We do _not_ want to encode anything.
>
> By convention, if your CPU is ARM mode capable, then it must call the
> kernel in ARM mode, period.  Can't be simpler than that, right?  The
> bootloader shouldn't care if the kernel is itself ARM or Thumb, and the
> kernel shouldn't care if the bootloader is ARM or Thumb.
>
> If you have a real problem with this then let's hear it so it can be
> addressed.

I don't have a real *problem* with it, I just think since the kernel
can be compiled in both modes, but you may have some kind of remit to
not run ARM code (even if it's just a couple instructions at the
start) or some toolchain requirement or.. something.. it would be
entirely the bootloader's job to determine what mode it is in when it
jumps into the kernel. If you boot a Thumb bootloader and load an ARM
kernel, you'll get a crash. If you boot an ARM bootloader and a Thumb
kernel (with the current ARM instructions in head.S), you'll get a
crash. This is well defined, perfectly obvious, and I realise that..

But if you would need to micromanage the system such that they both
matched, or were perverse enough to build a Thumb bootloader and a
Thumb kernel, you might be counfounded that you actually created a
99.99% Thumb bootloader which switched to ARM mode before going into
the kernel, and a 99.99% Thumb kernel but needed to jump into Thumb
mode after the first part of the kernel code? It should be possible
(with that kernel config switch it would be) if you're into that kind
of configurability to build your Thumb bootloader, build your Thumb
kernel and have it never exit Thumb mode for any reason, even if it is
just for some few bytes of very minor bootstrapping.

The same perfectly obvious caveats apply, but it's possible. The
question is, if you could do it, how would you make sure bootloaders
booted the right way? I would hazard a guess that given your opinion
on it, it would be just as good if they all just crashed.

So let's just patch out the ARM parts if you're building a Thumb
kernel on a Thumb-only device, and assume it was in Thumb mode all
along if another subordinate option was selected.. that way the kernel
can automatically fulfill one side of the contract in that you
configured "build thumb2 kernel" and it would contain 0% normal arm
code for ARMv7-M (or -R) and be able to be prompted to do so if you
were building for ARMv6k/ARMv7-A. Ordinary people would not turn it
on.. whacky weirdos like you and me might do it for proof of concept
;)

All it needs is a bootloader able to fulfill it's side of the contract
(build a thumb2 bootloader..) and jump while in Thumb mode, which I am
100% certain exists. It'd be a 20 line patch maximum to make U-Boot do
this (it already can build as Thumb2 I think.. if not, there are
patches around) through an environment variable and distros that knew
they built in Thumb mode could simply modify the variable in boot.scr
or so, and the jump would end up being done in Thumb. That way you
have to kick it, but no changes are required in the kernel format
(just in the packaging of the kernel and configuration of the
bootloader). Guessing would be a bad idea...

I still say considering you thought of a workable solution and your
problem is only the linker in use (seems like a bug report to the gas
guys is in order..) and there are probably ways to get around that
anyway, why not do it? I find it odd that Linux needs a "hard customer
requirement" for it to need it to be capable of doing it, if it's just
*possible*.

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Nicolas Pitre]

On Mon, 6 Aug 2012, Matt Sealey wrote:

> On Mon, Aug 6, 2012 at 5:36 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > On Mon, 6 Aug 2012, Matt Sealey wrote:
> >
> >> Right now there's already a fixed entry point - ARM kernels are fixed
> >> entry point of relative 0 to the start of the loaded image. Okay, so
> >> we can't format a totally valid ELF header, but in the event that we
> >> want to encode all this information yet again, why reinvent the wheel?
> >
> > That's the point.  We do _not_ want to encode anything.
> >
> > By convention, if your CPU is ARM mode capable, then it must call the
> > kernel in ARM mode, period.  Can't be simpler than that, right?  The
> > bootloader shouldn't care if the kernel is itself ARM or Thumb, and the
> > kernel shouldn't care if the bootloader is ARM or Thumb.
> >
> > If you have a real problem with this then let's hear it so it can be
> > addressed.
> 
> I don't have a real *problem* with it, I just think since the kernel
> can be compiled in both modes, but you may have some kind of remit to
> not run ARM code (even if it's just a couple instructions at the
> start) or some toolchain requirement or.. something.. it would be
> entirely the bootloader's job to determine what mode it is in when it
> jumps into the kernel. If you boot a Thumb bootloader and load an ARM
> kernel, you'll get a crash. If you boot an ARM bootloader and a Thumb
> kernel (with the current ARM instructions in head.S), you'll get a
> crash. This is well defined, perfectly obvious, and I realise that..

And isn't it also rather silly?

History has told that the more "entirely the bootloader's job" we rely 
on, the less reliable and interoperable we become.  Because you now have 
to compile two kernels to fully validate your bootloader instead of only 
one, and some people won't bother if the second kernel doesn't match 
their own use case.

> But if you would need to micromanage the system such that they both
> matched, or were perverse enough to build a Thumb bootloader and a
> Thumb kernel, you might be counfounded that you actually created a
> 99.99% Thumb bootloader which switched to ARM mode before going into
> the kernel, and a 99.99% Thumb kernel but needed to jump into Thumb
> mode after the first part of the kernel code?

I'm pragmatic enough to fully accept this impurity.

The x86 kernel does boot in 16-bit mode even if nobody compiles 16-bit 
applications anymore.

> It should be possible (with that kernel config switch it would be) if 
> you're into that kind of configurability to build your Thumb 
> bootloader, build your Thumb kernel and have it never exit Thumb mode 
> for any reason, even if it is just for some few bytes of very minor 
> bootstrapping.

And what does this buy you?

> The same perfectly obvious caveats apply, but it's possible. The
> question is, if you could do it, how would you make sure bootloaders
> booted the right way? I would hazard a guess that given your opinion
> on it, it would be just as good if they all just crashed.

Whereas right now it just works because the kernel is always entered in 
ARM mode.

> So let's just patch out the ARM parts if you're building a Thumb
> kernel on a Thumb-only device, and assume it was in Thumb mode all
> along if another subordinate option was selected.. that way the kernel
> can automatically fulfill one side of the contract in that you
> configured "build thumb2 kernel" and it would contain 0% normal arm
> code for ARMv7-M (or -R) and be able to be prompted to do so if you
> were building for ARMv6k/ARMv7-A. Ordinary people would not turn it
> on.. whacky weirdos like you and me might do it for proof of concept
> ;)

No.  People will play with it as well and complain because their kernel 
doesn't boot anymore.  We have enough real bug reports not to add to 
them with futile ones.  And there are already way too many kernel config 
options.

Some other people will hardcode Thumb mode entry in their bootloaders.  
And if you wanted to compile your kernel in ARM mode for such a device 
then you'd have the perverse need for a shim in Thumb mode at the 
beginning of the kernel image to switch to ARM mode.

Whacky Weirdos can hack their kernel all they want.  That doesn't have 
to be sanctioned in mainline.

> All it needs is a bootloader able to fulfill it's side of the contract
> (build a thumb2 bootloader..) and jump while in Thumb mode, which I am
> 100% certain exists. It'd be a 20 line patch maximum to make U-Boot do
> this (it already can build as Thumb2 I think.. if not, there are
> patches around) through an environment variable and distros that knew
> they built in Thumb mode could simply modify the variable in boot.scr
> or so, and the jump would end up being done in Thumb. That way you
> have to kick it, but no changes are required in the kernel format
> (just in the packaging of the kernel and configuration of the
> bootloader). Guessing would be a bad idea...

The fewer contracts we have with bootloaders the better we are.  
Unfortunately those contracts are increasing in scope.  Let's avoid the 
really trivial cases if we can help it.

By mandating ARM mode, there is absolutely no guessing and no place for 
misinterpretation, buggy implementation, or other silliness of which 
bootloaders are full of.

The only good case for using Thumb mode in kernel entry is with those 
CPUs with no support for ARM mode.


Nicolas

kernel entry for thumb2-only cpus

[Russell King - ARM Linux]

+1 on everything Nicolas has said below.  Full violent agreement there.

On Mon, Aug 06, 2012 at 08:08:57PM -0400, Nicolas Pitre wrote:
> On Mon, 6 Aug 2012, Matt Sealey wrote:
> 
> > On Mon, Aug 6, 2012 at 5:36 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > > On Mon, 6 Aug 2012, Matt Sealey wrote:
> > >
> > >> Right now there's already a fixed entry point - ARM kernels are fixed
> > >> entry point of relative 0 to the start of the loaded image. Okay, so
> > >> we can't format a totally valid ELF header, but in the event that we
> > >> want to encode all this information yet again, why reinvent the wheel?
> > >
> > > That's the point.  We do _not_ want to encode anything.
> > >
> > > By convention, if your CPU is ARM mode capable, then it must call the
> > > kernel in ARM mode, period.  Can't be simpler than that, right?  The
> > > bootloader shouldn't care if the kernel is itself ARM or Thumb, and the
> > > kernel shouldn't care if the bootloader is ARM or Thumb.
> > >
> > > If you have a real problem with this then let's hear it so it can be
> > > addressed.
> > 
> > I don't have a real *problem* with it, I just think since the kernel
> > can be compiled in both modes, but you may have some kind of remit to
> > not run ARM code (even if it's just a couple instructions at the
> > start) or some toolchain requirement or.. something.. it would be
> > entirely the bootloader's job to determine what mode it is in when it
> > jumps into the kernel. If you boot a Thumb bootloader and load an ARM
> > kernel, you'll get a crash. If you boot an ARM bootloader and a Thumb
> > kernel (with the current ARM instructions in head.S), you'll get a
> > crash. This is well defined, perfectly obvious, and I realise that..
> 
> And isn't it also rather silly?
> 
> History has told that the more "entirely the bootloader's job" we rely 
> on, the less reliable and interoperable we become.  Because you now have 
> to compile two kernels to fully validate your bootloader instead of only 
> one, and some people won't bother if the second kernel doesn't match 
> their own use case.
> 
> > But if you would need to micromanage the system such that they both
> > matched, or were perverse enough to build a Thumb bootloader and a
> > Thumb kernel, you might be counfounded that you actually created a
> > 99.99% Thumb bootloader which switched to ARM mode before going into
> > the kernel, and a 99.99% Thumb kernel but needed to jump into Thumb
> > mode after the first part of the kernel code?
> 
> I'm pragmatic enough to fully accept this impurity.
> 
> The x86 kernel does boot in 16-bit mode even if nobody compiles 16-bit 
> applications anymore.
> 
> > It should be possible (with that kernel config switch it would be) if 
> > you're into that kind of configurability to build your Thumb 
> > bootloader, build your Thumb kernel and have it never exit Thumb mode 
> > for any reason, even if it is just for some few bytes of very minor 
> > bootstrapping.
> 
> And what does this buy you?
> 
> > The same perfectly obvious caveats apply, but it's possible. The
> > question is, if you could do it, how would you make sure bootloaders
> > booted the right way? I would hazard a guess that given your opinion
> > on it, it would be just as good if they all just crashed.
> 
> Whereas right now it just works because the kernel is always entered in 
> ARM mode.
> 
> > So let's just patch out the ARM parts if you're building a Thumb
> > kernel on a Thumb-only device, and assume it was in Thumb mode all
> > along if another subordinate option was selected.. that way the kernel
> > can automatically fulfill one side of the contract in that you
> > configured "build thumb2 kernel" and it would contain 0% normal arm
> > code for ARMv7-M (or -R) and be able to be prompted to do so if you
> > were building for ARMv6k/ARMv7-A. Ordinary people would not turn it
> > on.. whacky weirdos like you and me might do it for proof of concept
> > ;)
> 
> No.  People will play with it as well and complain because their kernel 
> doesn't boot anymore.  We have enough real bug reports not to add to 
> them with futile ones.  And there are already way too many kernel config 
> options.
> 
> Some other people will hardcode Thumb mode entry in their bootloaders.  
> And if you wanted to compile your kernel in ARM mode for such a device 
> then you'd have the perverse need for a shim in Thumb mode at the 
> beginning of the kernel image to switch to ARM mode.
> 
> Whacky Weirdos can hack their kernel all they want.  That doesn't have 
> to be sanctioned in mainline.
> 
> > All it needs is a bootloader able to fulfill it's side of the contract
> > (build a thumb2 bootloader..) and jump while in Thumb mode, which I am
> > 100% certain exists. It'd be a 20 line patch maximum to make U-Boot do
> > this (it already can build as Thumb2 I think.. if not, there are
> > patches around) through an environment variable and distros that knew
> > they built in Thumb mode could simply modify the variable in boot.scr
> > or so, and the jump would end up being done in Thumb. That way you
> > have to kick it, but no changes are required in the kernel format
> > (just in the packaging of the kernel and configuration of the
> > bootloader). Guessing would be a bad idea...
> 
> The fewer contracts we have with bootloaders the better we are.  
> Unfortunately those contracts are increasing in scope.  Let's avoid the 
> really trivial cases if we can help it.
> 
> By mandating ARM mode, there is absolutely no guessing and no place for 
> misinterpretation, buggy implementation, or other silliness of which 
> bootloaders are full of.
> 
> The only good case for using Thumb mode in kernel entry is with those 
> CPUs with no support for ARM mode.
> 
> 
> Nicolas

kernel entry for thumb2-only cpus

[Matt Sealey]

On Mon, Aug 6, 2012 at 7:08 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Mon, 6 Aug 2012, Matt Sealey wrote:
>
>> I don't have a real *problem* with it, I just think since the kernel
>> can be compiled in both modes, but you may have some kind of remit to
>> not run ARM code (even if it's just a couple instructions at the
>> start) or some toolchain requirement or.. something.. it would be
>> entirely the bootloader's job to determine what mode it is in when it
>> jumps into the kernel. If you boot a Thumb bootloader and load an ARM
>> kernel, you'll get a crash. If you boot an ARM bootloader and a Thumb
>> kernel (with the current ARM instructions in head.S), you'll get a
>> crash. This is well defined, perfectly obvious, and I realise that..
>
> And isn't it also rather silly?

No?

> History has told that the more "entirely the bootloader's job" we rely
> on, the less reliable and interoperable we become.  Because you now have
> to compile two kernels to fully validate your bootloader instead of only
> one, and some people won't bother if the second kernel doesn't match
> their own use case.

That is true from a testing standpoint, and I guess I do agree with yourself and
Russell in this regard. But I do also think the chain of trust from
the bootloader
to kernel is important. I am in full violent disagreement with the
idea that doing
pin mux change operations based on a device tree description is both a necessity
or a requirement in Linux, especially as changing pin muxing this "late" in boot
is something a hardware designer would flip their lid over.

Sure, on some boards this is a requirement because the bootloaders do not do
the right thing, but in actual fact some of these boards (MX51 Babbage, MX6
SabreLite for example) cannot boot a device tree from their shipped bootloaders
anyway. To boot these boards from device tree you need to actually update U-Boot
or provide another compatible, capable bootloader, and in this case this support
should actually be provided in the bootloader (mux everything before
the kernel is
even in memory, and you reduce your device tree sizes by a factor of 3). The
idea that Linux should reinitialize everything anyway just to be sure is both
redundant, and means you're slowing down boot times and potentially causing
odd behavior (electrically) on boards with reset lines flailing around
during a mux
change, and GPIO directions changing on the fly for no good reason.

The device tree model has to imply some level of trust that the
configuration you
have in the device tree is exactly what you get on the board when the kernel has
booted. If that means adding pinctrl information to each device to
make sure it is
set up properly, then so be it. That said..

At Genesi we are proceeding with device tree support for all our platforms and
operating from an assumption that since we control our bootloader (it is U-Boot
after all, so not heavily..), we get to dictate what goes in our
device tree and we
will make a hard and fast stand against ANY pin mux changes going on inside the
kernel. And you guys will have to trust that we sat down and spent weeks and
weeks and weeks testing that this is actually the case before we committed
anything.

Just because there is a ton of absolutely awful, broken firmware code out there
doesn't mean it can and will always be the case, and Linux policy should not
be dictated on an architecture basis on a few bad eggs, especially if it means
developers in the big wide world have to jump through hoops. Surely it is just
as good for Linux to loudly advocate the correct solutions in firmware and
implement the workarounds anyway, like a device quirk, rather than just out and
out say "firmwares suck, ignore what they did, do it again. We don't trust them
and never will."

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Nicolas Pitre]

On Tue, 7 Aug 2012, Matt Sealey wrote:

> On Mon, Aug 6, 2012 at 7:08 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > On Mon, 6 Aug 2012, Matt Sealey wrote:
> >
> >> I don't have a real *problem* with it, I just think since the kernel
> >> can be compiled in both modes, but you may have some kind of remit to
> >> not run ARM code (even if it's just a couple instructions at the
> >> start) or some toolchain requirement or.. something.. it would be
> >> entirely the bootloader's job to determine what mode it is in when it
> >> jumps into the kernel. If you boot a Thumb bootloader and load an ARM
> >> kernel, you'll get a crash. If you boot an ARM bootloader and a Thumb
> >> kernel (with the current ARM instructions in head.S), you'll get a
> >> crash. This is well defined, perfectly obvious, and I realise that..
> >
> > And isn't it also rather silly?
> 
> No?
> 
> > History has told that the more "entirely the bootloader's job" we rely
> > on, the less reliable and interoperable we become.  Because you now have
> > to compile two kernels to fully validate your bootloader instead of only
> > one, and some people won't bother if the second kernel doesn't match
> > their own use case.
> 
> That is true from a testing standpoint, and I guess I do agree with 
> yourself and Russell in this regard. But I do also think the chain of 
> trust from the bootloader to kernel is important. I am in full violent 
> disagreement with the idea that doing pin mux change operations based 
> on a device tree description is both a necessity or a requirement in 
> Linux, especially as changing pin muxing this "late" in boot is 
> something a hardware designer would flip their lid over.
[...]

I don't think it is reasonable to defer every pin mux setup to the 
kernel.  The bootloader obviously has to set those things up.  However 
the bootloader is way gone when it is time to perform CPU idle 
and suspend and operations.  This is when the DT representation is most 
useful.

> Sure, on some boards this is a requirement because the bootloaders do 
> not do the right thing, but in actual fact some of these boards (MX51 
> Babbage, MX6 SabreLite for example) cannot boot a device tree from 
> their shipped bootloaders anyway. To boot these boards from device 
> tree you need to actually update U-Boot or provide another compatible, 
> capable bootloader, and in this case this support should actually be 
> provided in the bootloader (mux everything before the kernel is even 
> in memory, and you reduce your device tree sizes by a factor of 3).

I do not disagree with you here.

> The idea that Linux should reinitialize everything anyway just to be 
> sure is both redundant, and means you're slowing down boot times and 
> potentially causing odd behavior (electrically) on boards with reset 
> lines flailing around during a mux change, and GPIO directions 
> changing on the fly for no good reason.

But if Linux is re-initializing everything identically to the bootloader 
then the hardware should see a no-op, no?  In any case, I'm not the 
right person to discuss pinmux design issues.  If you have concerns 
about redundant initialization or similar issues then I strongly 
suggest you start a new thread to initiate a discussion on that.

> The device tree model has to imply some level of trust that the 
> configuration you have in the device tree is exactly what you get on 
> the board when the kernel has booted. If that means adding pinctrl 
> information to each device to make sure it is set up properly, then so 
> be it. That said..

Yes, this is diverging from the original topic.

> At Genesi we are proceeding with device tree support for all our platforms and
> operating from an assumption that since we control our bootloader (it is U-Boot
> after all, so not heavily..), we get to dictate what goes in our
> device tree and we
> will make a hard and fast stand against ANY pin mux changes going on inside the
> kernel. And you guys will have to trust that we sat down and spent weeks and
> weeks and weeks testing that this is actually the case before we committed
> anything.

Note that I previously opposed the idea of any hard linkage between the 
bootloader and the device tree content.  If you are spending so much 
time validating your DT, you might feel so confident as to not allow it 
to be updatable unless the whole bootloader is also updated.  And that's 
wrong.  Please make your DTB into a separate flash area, separate from 
the boot loader so it can easily be updated without any risk of breaking 
the device.  Just in case...

> Just because there is a ton of absolutely awful, broken firmware code out there
> doesn't mean it can and will always be the case, and Linux policy should not
> be dictated on an architecture basis on a few bad eggs, especially if it means
> developers in the big wide world have to jump through hoops. Surely it is just
> as good for Linux to loudly advocate the correct solutions in firmware and
> implement the workarounds anyway, like a device quirk, rather than just out and
> out say "firmwares suck, ignore what they did, do it again. We don't trust them
> and never will."

I hear your frustration.  I'm glad you take your firmware work 
seriously.  But unless and until everyone else is like you, the kernel 
has to work defensively.  That means not giving too much freedom to the 
bootloader in order to preserve good interoperability.  That certainly 
includes _not_ allowing variations in the processor mode for booting the 
kernel.


Nicolas

kernel entry for thumb2-only cpus

[Dave Martin]

On Mon, Aug 06, 2012 at 12:32:14AM -0400, Nicolas Pitre wrote:
> On Sun, 5 Aug 2012, Uwe Kleine-K?nig wrote:
> 
> > On Fri, Aug 03, 2012 at 12:45:32PM +0100, Will Deacon wrote:
> > > On Fri, Aug 03, 2012 at 10:04:52AM +0100, Uwe Kleine-K?nig wrote:
> > > > Hello,
> > > 
> > > Hi Uwe,
> > > 
> > > > in both arch/arm/kernel/head.S and arch/arm/kernel/head-nommu.S we have
> > > > 
> > > > 	.arm
> > > >  THUMB( adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> > > >  THUMB( bx      r9              )       @ If this is a Thumb-2 kernel,
> > > >  THUMB( .thumb                  )       @ switch to Thumb now.
> > > >  THUMB(1:                       )
> > > > 
> > > > as first instructions at the entry point. This is a problem for
> > > > thumb2-only cpus (e.g. Cortex-M3).
> > > 
> > > Yup, Documentation/arm/Booting mentions this too.
> > > 
> > > > Up to now I commented out the first three lines for the Cortex-M3 port.
> > > > What should we do about that. There are two possibilities I see:
> > > > 
> > > >  a) introduce a kconfig symbol for thumb2-only builds and #ifdef the
> > > >     first three lines out if it is given.
> > > >  b) expect the bootloader to directly jump to the label 1.
> > > > 
> > > > The downside of a) is that a boot loader on a cpu that is capable to
> > > > execute the tradtional instructions would need to detect if the switch
> > > > to thumb is there or not and react accordingly. (In fact it needs to
> > > > distringuish three cases:
> > > > 
> > > >  - traditional kernel
> > > >  - thumb2 kernel with ARM entry
> > > >  - thumb2 kernel without ARM entry
> > > > 
> > > > .) So I think b) is the more sensible option in the long run.
> > > > 
> > > > What do you think?
> > > 
> > > How about something like:
> > > 
> > > 
> > > diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
> > > index 835898e..9f07be2 100644
> > > --- a/arch/arm/kernel/head.S
> > > +++ b/arch/arm/kernel/head.S
> > > @@ -86,6 +86,7 @@
> > >         __HEAD
> > >  ENTRY(stext)
> > >  
> > > + THUMB(        .inst   0xe200e004      )       @ ARM: and lr, r0, #4 T2: b 1f
> > >   THUMB(        adr     r9, BSYM(1f)    )       @ Kernel is always entered in ARM.
> > >   THUMB(        bx      r9              )       @ If this is a Thumb-2 kernel,
> > >   THUMB(        .thumb                  )       @ switch to Thumb now.
> > Great idea, but it doesn't work as suggested. My compiler already barfs
> > on the .arm above:
> 
> Let's stop splitting hairs.
> 
> A Cortex-M kernel simply won't run on anything else.  It certainly won't 
> run on ARM mode capable processors.  So in this case just make the 
> kernel entry point Thumb mode only by #ifdef'ing out the .arm part.

This was exactly my view when originally adding that code.

Previously, there was no well-defined way for a bootloader to know
whether the target kernel was ARM or Thumb and this had to be separately
configured.  This just caused a compatibility mess.

To fix this, we now have a fixed boot protocol: on a CPU supporting ARM
state, you must enter the kernel in ARM state, period.  This means there
is nothing for the bootloader to get wrong.  If a single bootloader
image is required across CPUs with and without ARM state, the bootloader
must detect the CPU and enter the kernel in the appropriate way.

The unconditionalness of the veneer sequence was an omission caused by
the fact that there was no real upstream support for ARM-less v7 CPUs
at the time.

For ARM-less CPUs, I would go for the #ifdef approach.


The only potential problem comes from the fact that the boot protocol
groups CPUs into two mutually exclusive sets: those with ARM state and
those without.  I single image cannot therefore run on both:

 a) building a single image that is bootable on ARM-less (M class)
    and AR class.  Those architecture families are just too different
    for this to seem realistic.

 b) building a single image that is bootable on current -AR CPUs
    (with ARM state) and hypothetical future -AR CPUs without ARM
    state.

Both of these seem pretty speculative.

Unless we have a need for these right now I would much rather that a
bootloader which does not follow arm/Booting fails instead of silently
working.

Cheers
---Dave

kernel entry for thumb2-only cpus

[Russell King - ARM Linux]

On Tue, Aug 07, 2012 at 05:34:15PM -0500, Matt Sealey wrote:
> Just because there is a ton of absolutely awful, broken firmware code out there
> doesn't mean it can and will always be the case, and Linux policy should not
> be dictated on an architecture basis on a few bad eggs, especially if it means
> developers in the big wide world have to jump through hoops. Surely it is just
> as good for Linux to loudly advocate the correct solutions in firmware and
> implement the workarounds anyway, like a device quirk, rather than just out and
> out say "firmwares suck, ignore what they did, do it again. We don't trust them
> and never will."

Sadly, firmware developers have taught us time and time and time and time
again that they can not be trusted.  You may be the single one who can be
trusted to validate their stuff properly, but you would be in a severe
minority if that is true.

Many firmware developers do the barest minimum that's required.  Once their
job is done, that's the end of the development cycle and nothing further
happens, not even for bugs.

We've seen this time and time again.  We see it with corrupted ATAG lists,
we see it with bad memory tags passed to the kernel that the kernel has to
then screw around with to fix up the broken firmware developers crap.
Let me show you the crappy workarounds that we have:

static void __init
fixup_clep7312(struct tag *tags, char **cmdline, struct meminfo *mi)
{
        mi->nr_banks=1;
        mi->bank[0].start = 0xc0000000;
        mi->bank[0].size = 0x01000000;
}
static void __init
fixup_edb7211(struct tag *tags, char **cmdline, struct meminfo *mi)
{
        /*
         * Bank start addresses are not present in the information
         * passed in from the boot loader.  We could potentially
         * detect them, but instead we hard-code them.
         *
         * Banks sizes _are_ present in the param block, but we're
         * not using that information yet.
         */
        mi->bank[0].start = 0xc0000000;
        mi->bank[0].size = 8*1024*1024;
        mi->bank[1].start = 0xc1000000;
        mi->bank[1].size = 8*1024*1024;
        mi->nr_banks = 2;
}
static void __init
fortunet_fixup(struct tag *tags, char **cmdline, struct meminfo *mi)
{
...
        memmap.bank[0].size = ip->ram_size;
        *mi = memmap;
}
static void __init mahimahi_fixup(struct tag *tags, char **cmdline,
                                  struct meminfo *mi)
{
        mi->nr_banks = 2;
        mi->bank[0].start = PHYS_OFFSET;
        mi->bank[0].node = PHYS_TO_NID(PHYS_OFFSET);
        mi->bank[0].size = (219*1024*1024);
        mi->bank[1].start = MSM_HIGHMEM_BASE;
        mi->bank[1].node = PHYS_TO_NID(MSM_HIGHMEM_BASE);
        mi->bank[1].size = MSM_HIGHMEM_SIZE;
}
static void __init msm7x30_fixup(struct tag *tag, char **cmdline,
                struct meminfo *mi)
{
        for (; tag->hdr.size; tag = tag_next(tag))
                if (tag->hdr.tag == ATAG_MEM && tag->u.mem.start == 0x200000) {
                        tag->u.mem.start = 0;
                        tag->u.mem.size += SZ_2M;
                }
}
static void __init msm8960_fixup(struct tag *tag, char **cmdline,
                struct meminfo *mi)
{
        for (; tag->hdr.size; tag = tag_next(tag))
                if (tag->hdr.tag == ATAG_MEM &&
                                tag->u.mem.start == 0x40200000) {
                        tag->u.mem.start = 0x40000000;
                        tag->u.mem.size += SZ_2M;
                }
}
static void __init msm8x60_fixup(struct tag *tag, char **cmdline,
                struct meminfo *mi)
{
        for (; tag->hdr.size; tag = tag_next(tag))
                if (tag->hdr.tag == ATAG_MEM &&
                                tag->u.mem.start == 0x40200000) {
                        tag->u.mem.start = 0x40000000;
                        tag->u.mem.size += SZ_2M;
                }
}
static void __init sapphire_fixup(struct tag *tags, char **cmdline,
                                  struct meminfo *mi)
{
        int smi_sz = parse_tag_smi((const struct tag *)tags);

        mi->nr_banks = 1;
        mi->bank[0].start = PHYS_OFFSET;
        mi->bank[0].node = PHYS_TO_NID(PHYS_OFFSET);
        if (smi_sz == 32) {
                mi->bank[0].size = (84*1024*1024);
        } else if (smi_sz == 64) {
                mi->bank[0].size = (101*1024*1024);
        } else {
                /* Give a default value when not get smi size */
                smi_sz = 64;
                mi->bank[0].size = (101*1024*1024);
        }
}
static void __init trout_fixup(struct tag *tags, char **cmdline,
                               struct meminfo *mi)
{
        mi->nr_banks = 1;
        mi->bank[0].start = PHYS_OFFSET;
        mi->bank[0].size = (101*1024*1024);
}
static void __init fixup_corgi(struct tag *tags, char **cmdline,
                               struct meminfo *mi)
{
        sharpsl_save_param();
        mi->nr_banks=1;
        mi->bank[0].start = 0xa0000000;
        if (machine_is_corgi())
                mi->bank[0].size = (32*1024*1024);
        else
                mi->bank[0].size = (64*1024*1024);
}
void __init eseries_fixup(struct tag *tags, char **cmdline, struct meminfo *mi)
{
        mi->nr_banks=1;
        mi->bank[0].start = 0xa0000000;
        if (machine_is_e800())
                mi->bank[0].size = (128*1024*1024);
        else
                mi->bank[0].size = (64*1024*1024);
}
static void __init fixup_poodle(struct tag *tags, char **cmdline,
                                struct meminfo *mi)
{
        sharpsl_save_param();
        mi->nr_banks=1;
        mi->bank[0].start = 0xa0000000;
        mi->bank[0].size = (32*1024*1024);
}
static void __init spitz_fixup(struct tag *tags, char **cmdline,
                               struct meminfo *mi)
{
        sharpsl_save_param();
        mi->nr_banks = 1;
        mi->bank[0].start = 0xa0000000;
        mi->bank[0].size = (64*1024*1024);
}
static void __init fixup_tosa(struct tag *tags, char **cmdline,
                              struct meminfo *mi)
{
        sharpsl_save_param();
        mi->nr_banks=1;
        mi->bank[0].start = 0xa0000000;
        mi->bank[0].size = (64*1024*1024);
}
static void __init smdk2413_fixup(struct tag *tags, char **cmdline,
                                  struct meminfo *mi)
{
        if (tags != phys_to_virt(S3C2410_SDRAM_PA + 0x100)) {
                mi->nr_banks=1;
                mi->bank[0].start = 0x30000000;
                mi->bank[0].size = SZ_64M;
        }
}
static void __init vstms_fixup(struct tag *tags, char **cmdline,
                               struct meminfo *mi)
{
        if (tags != phys_to_virt(S3C2410_SDRAM_PA + 0x100)) {
                mi->nr_banks=1;
                mi->bank[0].start = 0x30000000;
                mi->bank[0].size = SZ_64M;
        }
}

Spot a pattern there?  The one which stands out to me is that boot loaders
can not get the trivial task of passing the simple information about where
the RAM is in the system to the kernel right.

Many boot loaders for _years_ have not been able to get the very very
trivially simple issue of passing the right machine ID value in r1 to the
kernel either.

History has taught us time and time and time again that firmware can
not be trusted in *any* *way* and we have plenty of proof to back that
up.

Have we tried to push the onus back on firmware people?  I've tried damned
hard to the extent of preventing some of these work-arounds getting into
mainline, but the sole result of that was that mainline would not work on
those platforms; it didn't make a blind bit of difference to the firmware
on the platforms.  It didn't magically cause the firmware to get fixed in
any way.  We just ended up with a detrimental situation to everyone because
mainline just didn't work on those platforms.

So please, stop telling us what we should and shouldn't do with firmware.
We've learnt by the action of firmware developers how to treat them, and
that's how we're going to treat them.  They have earned our distrust.  In
order for that to change, they must change, and they must _earn_ our trust
that they _can_ be trusted to do a good job.  Until then...

kernel entry for thumb2-only cpus

[Matt Sealey]

On Wed, Aug 8, 2012 at 12:04 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Tue, 7 Aug 2012, Matt Sealey wrote:
>
>> On Mon, Aug 6, 2012 at 7:08 PM, Nicolas Pitre <nico@fluxnic.net> wrote:
>> > On Mon, 6 Aug 2012, Matt Sealey wrote:
>
>> lines flailing around during a mux change, and GPIO directions
>> changing on the fly for no good reason.
>
> But if Linux is re-initializing everything identically to the bootloader
> then the hardware should see a no-op, no?

Not necessarily.. it is most likely implementation dependent, I'd think.
It also depends on what is outside the chip.

> right person to discuss pinmux design issues.  If you have concerns
> about redundant initialization or similar issues then I strongly
> suggest you start a new thread to initiate a discussion on that.

Been doing that since 2005, still going strong..

> Note that I previously opposed the idea of any hard linkage between the
> bootloader and the device tree content.  If you are spending so much
> time validating your DT, you might feel so confident as to not allow it
> to be updatable unless the whole bootloader is also updated.

The DTB is seperate from the firmware. But the firmware does all the pinmuxing
long before the DTB is loaded - that way the DTB does the correct thing and
in the worst case describes only the currently configured and ready-to-operate
hardware (or in the best case, describes hardware the bootloader is currently
using..).

On OF since every node tends to have a driver backed to it, you just write
small stub drivers that pinmux then add it to the tree. U-Boot and other FW
are more clunky but the same effect can be achieved - just make sure every
entry you commit to the Linux arch/arm/boot/dts tree definitely has it's pinmux
set up.

There are good reasons to pinmux stuff at runtime - configurable hardware such
as on my HP Touchpad, the headphone port is also a UART. It's creepy, but
you may want to listen to audio and then switch to debugging - the microphone
part is always input (uart rx on host side) and the left audio is
output (uart tx),
and ground is ground. The right audio pin is not used in UART. You don't want
to reboot in the middle of debugging an app that specifically uses the headphone
and with correct board design and prudent use by the developer it's safe.. but
not all boards are designed that way. One thing we can change is the software
to work around hardware. We can make sure DTBs describe the breakages well
and make sure that FW solves all the problems it can before Linux gets to
it (for instance ARM errata settings in SCU, PL310, aux cp registers which would
otherwise be done in cpu init once at boot, need to be done before running any
code so FW should do them).

But as we just saw when submitting our device tree, we got a nit from
the maintainer
that we don't set up pinctrl - and that pinctrl setup would be
required in case any
device drivers needed it. I'm saying no; pinctrl shouldn't be
*required* by a platform,
it should be optional for all those where it's impossible to entertain
it or where there's
some kind of late-hitting errata that nobody could have predicted
(like the MX51 SPI
chipselect bug) and the collaboration between updating U-Boot and Linux to match
using GPIO for chipselects instead of letting the chip handle it is
too much cost or
effort compared to adding 2 entries to a device tree and updating Linux.

But not 100 entries in device tree, please, and no hard requirements..

> wrong.  Please make your DTB into a separate flash area, separate from
> the boot loader so it can easily be updated without any risk of breaking
> the device.  Just in case...

Our DTB is on the filesystem, ext2 or ext3 or ext4 or vfat. SD card, PATA,
SATA or USB, or even NFS or TFTP. You could even take an SD card, put
it in a Windows box, and copy a new one in place, no Linux required. It's
totally userspace/user friendly. We made absolutely sure of it.

Thanks,

I'll leave the discussion for the other thread I spawned from this ;)

-- 
Matt Sealey <matt@genesi-usa.com>
Product Development Analyst, Genesi USA, Inc.

kernel entry for thumb2-only cpus

[Nicolas Pitre]

On Wed, 8 Aug 2012, Matt Sealey wrote:

> On Wed, Aug 8, 2012 at 12:04 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> > Note that I previously opposed the idea of any hard linkage between the
> > bootloader and the device tree content.  If you are spending so much
> > time validating your DT, you might feel so confident as to not allow it
> > to be updatable unless the whole bootloader is also updated.
> 
> The DTB is seperate from the firmware. But the firmware does all the pinmuxing
> long before the DTB is loaded - that way the DTB does the correct thing and
> in the worst case describes only the currently configured and ready-to-operate
> hardware (or in the best case, describes hardware the bootloader is currently
> using..).

I do not disagree with that.  What's the problem?

> There are good reasons to pinmux stuff at runtime - configurable hardware such
> as on my HP Touchpad, the headphone port is also a UART. It's creepy, but
> you may want to listen to audio and then switch to debugging

Exact.  There are cases like that where changing pin muxing at run time 
is necessary.  We therefore have a subsystem in Linux to deal with that 
in a common way.

But if nothing has to change at run time then just let the boot loader 
set it up and be done with it... if your boot loader isn't buggy that 
is.

I'm always fighting against too much reliance on DT especially for stuff 
that the code can easily probe at run time.  I'm all for _reducing_ the 
dependencies between all the moving pieces i.e. the firmware, the 
bootloader, the DTB and the kernel.  Interactions must be kept to a 
minimum.  Hence my stance on the fixed CPU mode for booting Linux.

> But as we just saw when submitting our device tree, we got a nit from 
> the maintainer that we don't set up pinctrl - and that pinctrl setup 
> would be required in case any device drivers needed it.

And what's wrong with that statement?  The key point here is "in case 
device drivers need it".  If they don't, you don't, right?  I don't 
think you should provide information in your DTB if it is never going to 
be useful.

> I'm saying no; pinctrl shouldn't be *required* by a platform, it 
> should be optional for all those where it's impossible to entertain it 
> or where there's some kind of late-hitting errata that nobody could 
> have predicted (like the MX51 SPI chipselect bug) and the 
> collaboration between updating U-Boot and Linux to match using GPIO 
> for chipselects instead of letting the chip handle it is too much cost 
> or effort compared to adding 2 entries to a device tree and updating 
> Linux.
> 
> But not 100 entries in device tree, please, and no hard requirements..

Again, no disagreement from me on this.


Nicolas