ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

Hello,

with the same toolchain, only changing from gcc6.3.0 to gcc7.1.0, the
kernel does not boot: using xz the decompression fails with
"XZ-compressed data is corrupt" / XZ_DATA_ERROR.

I think it might be this line in lib/xz/xz_dec_stream.c

    s->vli |= (vli_type)(byte & 0x7F) << s->pos;

Here gcc6 is using __aeabi_llsl

Looking at the disassembled sources, do you see evident issues in dec_vli()?
Kernel is built optimized for size. I am uploading the logs and huge
build dirs [1]

Pls look under /disassembled. Suffix is the gcc version.

decompress6.S -> built with  gcc6
decompress7.S -> built with  gcc7

I am puzzled because the same toolchain produces valid kernel for
qemuarm versatile-926ejs.

Finally, a short test with gzip compressed kernel shows it does not
boot as well so maybe the issue is another. I don't have serial here
now but I'll take more logs if necessary.

Thanks
Andrea

[1] https://github.com/andrea-adami/gcc7-kernel-debug

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Robert Jarzmik]

Andrea Adami <andrea.adami@gmail.com> writes:

> Hello,

Hi Andrea,

I have the same report on userspace side on buildroot from Petr [1], which
triggers endless segfaults in userspace (init) with gcc7 while everything is
fine with gcc6. I have confirmation on my test farm the problem happens as well.

Since debugging in userspace is far easier, I would suggest attacking the debug
with Petr on userspace side, and once sorted out, come back to kernel side.

Cheers.

--
Robert

[1] Mail from Petr
From: Petr Cvek <petrcvekcz@gmail.com>
Subject: cross-gcc
To: robert.jarzmik at free.fr
Date: Wed, 12 Jul 2017 02:37:35 +0200 (1 week, 1 day, 6 hours ago)
Message-ID: <ea45c82f-35a8-a908-d199-c72c0c801340@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1

Hi,

do you still have a PXA27x board? I'm trying to build a rootfs with a new GCC
(7.1 from crosstool-ng snapshot a few days ago, buildroot same way) and it keeps
segfaulting (busybox, sshd, udevd, Xorg). Do you have the same problem? (seems
it does not care for xscale/iwmmxt neither O1/O2/Os, maybe on O0).

The GCC 6.x was fine and if i will need to make a bug report I think I will need
confirmation.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
> Andrea Adami <andrea.adami@gmail.com> writes:
>
>> Hello,
>
> Hi Andrea,
>
> I have the same report on userspace side on buildroot from Petr [1], which
> triggers endless segfaults in userspace (init) with gcc7 while everything is
> fine with gcc6. I have confirmation on my test farm the problem happens as well.
>
> Since debugging in userspace is far easier, I would suggest attacking the debug
> with Petr on userspace side, and once sorted out, come back to kernel side.
>
> Cheers.
>
> --
> Robert


Hello,

sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.

Andrea


>
> [1] Mail from Petr
> From: Petr Cvek <petrcvekcz@gmail.com>
> Subject: cross-gcc
> To: robert.jarzmik at free.fr
> Date: Wed, 12 Jul 2017 02:37:35 +0200 (1 week, 1 day, 6 hours ago)
> Message-ID: <ea45c82f-35a8-a908-d199-c72c0c801340@gmail.com>
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1
>
> Hi,
>
> do you still have a PXA27x board? I'm trying to build a rootfs with a new GCC
> (7.1 from crosstool-ng snapshot a few days ago, buildroot same way) and it keeps
> segfaulting (busybox, sshd, udevd, Xorg). Do you have the same problem? (seems
> it does not care for xscale/iwmmxt neither O1/O2/Os, maybe on O0).
>
> The GCC 6.x was fine and if i will need to make a bug report I think I will need
> confirmation.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Aaro Koskinen]

Hi,

On Sat, Sep 02, 2017 at 11:50:55PM +0200, Andrea Adami wrote:
> On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
> > Andrea Adami <andrea.adami@gmail.com> writes:
> > I have the same report on userspace side on buildroot from Petr [1], which
> > triggers endless segfaults in userspace (init) with gcc7 while everything is
> > fine with gcc6. I have confirmation on my test farm the problem happens as well.
> >
> > Since debugging in userspace is far easier, I would suggest attacking the debug
> > with Petr on userspace side, and once sorted out, come back to kernel side.
> 
> sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
> Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.

Has anyone been able to debug this further? It seems there are also issues
on iop32x/n2100 with GCC 7.2 and binutils 2.29.1 - early boot hangs silently
and I cannot get earlyprintk to work. Compiling with GCC 6.4.0 it's OK.

A.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Robert Jarzmik]

Aaro Koskinen <aaro.koskinen@iki.fi> writes:

> Hi,
>
> On Sat, Sep 02, 2017 at 11:50:55PM +0200, Andrea Adami wrote:
>> On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
>> > Andrea Adami <andrea.adami@gmail.com> writes:
>> > I have the same report on userspace side on buildroot from Petr [1], which
>> > triggers endless segfaults in userspace (init) with gcc7 while everything is
>> > fine with gcc6. I have confirmation on my test farm the problem happens as well.
>> >
>> > Since debugging in userspace is far easier, I would suggest attacking the debug
>> > with Petr on userspace side, and once sorted out, come back to kernel side.
>> 
>> sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
>> Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.
>
> Has anyone been able to debug this further? It seems there are also issues
> on iop32x/n2100 with GCC 7.2 and binutils 2.29.1 - early boot hangs silently
> and I cannot get earlyprintk to work. Compiling with GCC 6.4.0 it's OK.
>
> A.
Hi Aaro,

I didn't find the time for GCC 7.2 testing.
Yet I have earlyprintk working on my platforms, you need :
 - on kernel command line for ttyS0 in my case :
   earlycon=early_pxa,mmio,0x40100000
 - in kernel config :
	CONFIG_SERIAL_8250=y
	CONFIG_SERIAL_8250_CONSOLE=y
	CONFIG_SERIAL_8250_PXA=y

Cheers.

-- 
Robert

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Aaro Koskinen]

Hi,

On Sat, Oct 14, 2017 at 11:57:30PM +0200, Robert Jarzmik wrote:
> Aaro Koskinen <aaro.koskinen@iki.fi> writes:
> > On Sat, Sep 02, 2017 at 11:50:55PM +0200, Andrea Adami wrote:
> >> On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
> >> > Andrea Adami <andrea.adami@gmail.com> writes:
> >> > I have the same report on userspace side on buildroot from Petr [1], which
> >> > triggers endless segfaults in userspace (init) with gcc7 while everything is
> >> > fine with gcc6. I have confirmation on my test farm the problem happens as well.
> >> >
> >> > Since debugging in userspace is far easier, I would suggest attacking the debug
> >> > with Petr on userspace side, and once sorted out, come back to kernel side.
> >> 
> >> sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
> >> Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.
> >
> > Has anyone been able to debug this further? It seems there are also issues
> > on iop32x/n2100 with GCC 7.2 and binutils 2.29.1 - early boot hangs silently
> > and I cannot get earlyprintk to work. Compiling with GCC 6.4.0 it's OK.
> 
> I didn't find the time for GCC 7.2 testing.
> Yet I have earlyprintk working on my platforms, you need :
>  - on kernel command line for ttyS0 in my case :
>    earlycon=early_pxa,mmio,0x40100000

It dies already during decompression so earlycon does not help much.

This seems to be simpler to debug/reproduce using busybox. Compiling
just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
produces failing xz decompression:

root at thecus-n2100:~$ echo foo > foo.txt
root at thecus-n2100:~$ xz foo.txt 
root at thecus-n2100:~$ ./busybox.gcc-7.2 xz -d foo.txt.xz 
xz: corrupted data
root at thecus-n2100:~$ ./busybox.gcc-6.4 xz -d foo.txt.xz 
root at thecus-n2100:~$ cat foo.txt
foo

gzip is no better:

root at thecus-n2100:~$ gzip foo.txt 
root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz 
gzip: crc error
root at thecus-n2100:~$ ./busybox.gcc-6.4 gzip -d foo.txt.gz 
root at thecus-n2100:~$ cat foo.txt
foo

With bigger files (e.g. compressed binary), gzip gets stuck and produces
an ever growing file.

This requires specific hardware to reproduce:

root at thecus-n2100:~$ cat /proc/cpuinfo 
processor       : 0
model name      : XScale-80219 rev 0 (v5l)
BogoMIPS        : 591.87
Features        : swp half thumb fastmult edsp 
CPU implementer : 0x69
CPU architecture: 5TE
CPU variant     : 0x0
CPU part        : 0x2e3
CPU revision    : 0

Hardware        : Thecus N2100
Revision        : 0000
Serial          : 0000000000000000

The same failing binary (busybox.gcc-7.2) seems to work fine on RPi 2...?!

Anyway it seems that GCC 7.2 is unsuitable for building Linux at least
for some ARM platforms.

A.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Russell King - ARM Linux]

On Sun, Oct 15, 2017 at 01:46:40PM +0300, Aaro Koskinen wrote:
> Hi,
> 
> It dies already during decompression so earlycon does not help much.
> 
> This seems to be simpler to debug/reproduce using busybox. Compiling
> just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
> produces failing xz decompression:
> 
> root at thecus-n2100:~$ echo foo > foo.txt
> root at thecus-n2100:~$ xz foo.txt 
> root at thecus-n2100:~$ ./busybox.gcc-7.2 xz -d foo.txt.xz 
> xz: corrupted data
> root at thecus-n2100:~$ ./busybox.gcc-6.4 xz -d foo.txt.xz 
> root at thecus-n2100:~$ cat foo.txt
> foo
> 
> gzip is no better:
> 
> root at thecus-n2100:~$ gzip foo.txt 
> root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz 
> gzip: crc error
> root at thecus-n2100:~$ ./busybox.gcc-6.4 gzip -d foo.txt.gz 
> root at thecus-n2100:~$ cat foo.txt
> foo
> 
> With bigger files (e.g. compressed binary), gzip gets stuck and produces
> an ever growing file.
> 
> This requires specific hardware to reproduce:
> 
> root at thecus-n2100:~$ cat /proc/cpuinfo 
> processor       : 0
> model name      : XScale-80219 rev 0 (v5l)
> BogoMIPS        : 591.87
> Features        : swp half thumb fastmult edsp 
> CPU implementer : 0x69
> CPU architecture: 5TE
> CPU variant     : 0x0
> CPU part        : 0x2e3
> CPU revision    : 0
> 
> Hardware        : Thecus N2100
> Revision        : 0000
> Serial          : 0000000000000000
> 
> The same failing binary (busybox.gcc-7.2) seems to work fine on RPi 2...?!
> 
> Anyway it seems that GCC 7.2 is unsuitable for building Linux at least
> for some ARM platforms.

What mode do you have for alignment handling?

cat /proc/cpu/alignment

Default for older CPUs is to "ignore" then, which basically means that
unaligned loads are rotated - in other words, the 32-bit naturally
aligned word is loaded and then rotated such that the addressed byte
is in bits 0-7.  This is standard behaviour for older CPUs - it's
how the CPU works when there's no trapping of unaligned accesses, and
it should really be the way that GCC expects these CPUs to work.

Since there have been moves to drop older architecture support from GCC,
I wouldn't be surprised if handling for pre-ARMv6 CPUs was bitrotting
and GCC was producing code that fails.

I also know that various folk have put out calls for help with tooling
for older CPUs (without which support gets dropped), but I don't think
anyone stepped up.

So... you might be able to work around the userspace issues by setting
the alignment fault handling to "fixup" but that really won't help the
decompressor - we have no ways to take any faults in the decompressor,
so we can't fix up an unaligned access.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Aaro Koskinen]

Hi,

On Sun, Oct 15, 2017 at 02:01:19PM +0100, Russell King - ARM Linux wrote:
> On Sun, Oct 15, 2017 at 01:46:40PM +0300, Aaro Koskinen wrote:
> > This seems to be simpler to debug/reproduce using busybox. Compiling
> > just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
> > produces failing xz decompression:
> > 
> > root at thecus-n2100:~$ echo foo > foo.txt
> > root at thecus-n2100:~$ xz foo.txt 
> > root at thecus-n2100:~$ ./busybox.gcc-7.2 xz -d foo.txt.xz 
> > xz: corrupted data

[...]

> What mode do you have for alignment handling?
> 
> cat /proc/cpu/alignment

Oh right, thanks, yes it seems GCC generates unaligned access:

root at thecus-n2100:~$ echo 0 > /proc/cpu/alignment 
root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz 
gzip: crc error
root at thecus-n2100:~$ echo 2 > /proc/cpu/alignment 
root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz 
root at thecus-n2100:~$ 

> Default for older CPUs is to "ignore" then, which basically means that
> unaligned loads are rotated - in other words, the 32-bit naturally
> aligned word is loaded and then rotated such that the addressed byte
> is in bits 0-7.  This is standard behaviour for older CPUs - it's
> how the CPU works when there's no trapping of unaligned accesses, and
> it should really be the way that GCC expects these CPUs to work.
> 
> Since there have been moves to drop older architecture support from GCC,
> I wouldn't be surprised if handling for pre-ARMv6 CPUs was bitrotting
> and GCC was producing code that fails.
> 
> I also know that various folk have put out calls for help with tooling
> for older CPUs (without which support gets dropped), but I don't think
> anyone stepped up.
> 
> So... you might be able to work around the userspace issues by setting
> the alignment fault handling to "fixup" but that really won't help the
> decompressor - we have no ways to take any faults in the decompressor,
> so we can't fix up an unaligned access.

Yes, the fixup is not really a solution. What is confusing is that I
was able to compile bootable kernels for OMAP1 boards, so there GCC is
compiling the decompressor correctly.

A.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Hi,

Dne 15.10.2017 v 12:46 Aaro Koskinen napsal(a):
> Hi,
> 
> On Sat, Oct 14, 2017 at 11:57:30PM +0200, Robert Jarzmik wrote:
>> Aaro Koskinen <aaro.koskinen@iki.fi> writes:
>>> On Sat, Sep 02, 2017 at 11:50:55PM +0200, Andrea Adami wrote:
>>>> On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
>>>>> Andrea Adami <andrea.adami@gmail.com> writes:
>>>>> I have the same report on userspace side on buildroot from Petr [1], which
>>>>> triggers endless segfaults in userspace (init) with gcc7 while everything is
>>>>> fine with gcc6. I have confirmation on my test farm the problem happens as well.
>>>>>
>>>>> Since debugging in userspace is far easier, I would suggest attacking the debug
>>>>> with Petr on userspace side, and once sorted out, come back to kernel side.
>>>>
>>>> sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
>>>> Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.
>>>
>>> Has anyone been able to debug this further? It seems there are also issues
>>> on iop32x/n2100 with GCC 7.2 and binutils 2.29.1 - early boot hangs silently
>>> and I cannot get earlyprintk to work. Compiling with GCC 6.4.0 it's OK.
>>
>> I didn't find the time for GCC 7.2 testing.
>> Yet I have earlyprintk working on my platforms, you need :
>>   - on kernel command line for ttyS0 in my case :
>>     earlycon=early_pxa,mmio,0x40100000
> 
> It dies already during decompression so earlycon does not help much.
> 
> This seems to be simpler to debug/reproduce using busybox. Compiling
> just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
> produces failing xz decompression:
> 

Does it fail too when compiled with gcc 7.2 and with march=armv5t and/or 
march=armv4t ?

> root at thecus-n2100:~$ echo foo > foo.txt
> root at thecus-n2100:~$ xz foo.txt
> root at thecus-n2100:~$ ./busybox.gcc-7.2 xz -d foo.txt.xz
> xz: corrupted data
> root at thecus-n2100:~$ ./busybox.gcc-6.4 xz -d foo.txt.xz
> root at thecus-n2100:~$ cat foo.txt
> foo
> 
> gzip is no better:
> 
> root at thecus-n2100:~$ gzip foo.txt
> root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz
> gzip: crc error
> root at thecus-n2100:~$ ./busybox.gcc-6.4 gzip -d foo.txt.gz
> root at thecus-n2100:~$ cat foo.txt
> foo
> 

Is it really compiled with armv5te ("readelf -n -A")? I've just found my 
old gcc 6.3 compiled some code with a default armv4t.

Petr

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

<cut>

Hi,

I am trying to debug further but I don't have any output on serial console...
I enabled:

CONFIG_DEBUG_LL=y
CONFIG_DEBUG_PXA_UART1=y
# CONFIG_DEBUG_ICEDCC is not set
# CONFIG_DEBUG_SEMIHOSTING is not set
# CONFIG_DEBUG_LL_UART_8250 is not set
# CONFIG_DEBUG_LL_UART_PL01X is not set
CONFIG_DEBUG_LL_INCLUDE="debug/8250.S"
CONFIG_DEBUG_UART_8250=y
CONFIG_DEBUG_UART_PHYS=0x40100000
CONFIG_DEBUG_UART_VIRT=0xf6200000
CONFIG_DEBUG_UART_8250_SHIFT=2
# CONFIG_DEBUG_UART_8250_WORD is not set
# CONFIG_DEBUG_UART_8250_PALMCHIP is not set
# CONFIG_DEBUG_UART_8250_FLOW_CONTROL is not set
CONFIG_UNCOMPRESS_INCLUDE="mach/uncompress.h"
CONFIG_EARLY_PRINTK=y

then I added #define DEBUG  to both head.S and decompress.c under
arm/boot/compressed.
I added earlyprintk to the cmdline args used by the kexec bootloader but nothing

That kernel does boot silently, just 'Uncompresing..' then
'XZ-compressed data is corrupt -- System halted'

I do have output from the Angel bootloader at 9600 and by the kernel
at 115200 so the uart is working...
Any hint?
Thanks
Andrea

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Dne 17.10.2017 v 10:02 Andrea Adami napsal(a):
> <cut>
> 
> Hi,
Hi

> 
> I am trying to debug further but I don't have any output on serial console...
> I enabled:
> 
> CONFIG_DEBUG_LL=y
> CONFIG_DEBUG_PXA_UART1=y
> # CONFIG_DEBUG_ICEDCC is not set
> # CONFIG_DEBUG_SEMIHOSTING is not set
> # CONFIG_DEBUG_LL_UART_8250 is not set
> # CONFIG_DEBUG_LL_UART_PL01X is not set
> CONFIG_DEBUG_LL_INCLUDE="debug/8250.S"
> CONFIG_DEBUG_UART_8250=y
> CONFIG_DEBUG_UART_PHYS=0x40100000
> CONFIG_DEBUG_UART_VIRT=0xf6200000
> CONFIG_DEBUG_UART_8250_SHIFT=2
> # CONFIG_DEBUG_UART_8250_WORD is not set
> # CONFIG_DEBUG_UART_8250_PALMCHIP is not set
> # CONFIG_DEBUG_UART_8250_FLOW_CONTROL is not set
> CONFIG_UNCOMPRESS_INCLUDE="mach/uncompress.h"
> CONFIG_EARLY_PRINTK=y
> 
> then I added #define DEBUG  to both head.S and decompress.c under
> arm/boot/compressed.
> I added earlyprintk to the cmdline args used by the kexec bootloader but nothing
> 
> That kernel does boot silently, just 'Uncompresing..' then
> 'XZ-compressed data is corrupt -- System halted'
> 
> I do have output from the Angel bootloader at 9600 and by the kernel
> at 115200 so the uart is working...
> Any hint?

If it is a problem with gcc (as Russell said before), there is no way 
you can fix it in the kernel. Only compiling it with an older gcc 
version can help.

Maybe you can try to compile it for armv4 code.

best regards,
Petr

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

On Tue, Oct 17, 2017 at 10:10 AM, Petr Cvek <petrcvekcz@gmail.com> wrote:
> Dne 17.10.2017 v 10:02 Andrea Adami napsal(a):
>>
>> <cut>
>>
>> Hi,
>
> Hi
>
>
>>
>> I am trying to debug further but I don't have any output on serial
>> console...
>> I enabled:
>>
>> CONFIG_DEBUG_LL=y
>> CONFIG_DEBUG_PXA_UART1=y
>> # CONFIG_DEBUG_ICEDCC is not set
>> # CONFIG_DEBUG_SEMIHOSTING is not set
>> # CONFIG_DEBUG_LL_UART_8250 is not set
>> # CONFIG_DEBUG_LL_UART_PL01X is not set
>> CONFIG_DEBUG_LL_INCLUDE="debug/8250.S"
>> CONFIG_DEBUG_UART_8250=y
>> CONFIG_DEBUG_UART_PHYS=0x40100000
>> CONFIG_DEBUG_UART_VIRT=0xf6200000
>> CONFIG_DEBUG_UART_8250_SHIFT=2
>> # CONFIG_DEBUG_UART_8250_WORD is not set
>> # CONFIG_DEBUG_UART_8250_PALMCHIP is not set
>> # CONFIG_DEBUG_UART_8250_FLOW_CONTROL is not set
>> CONFIG_UNCOMPRESS_INCLUDE="mach/uncompress.h"
>> CONFIG_EARLY_PRINTK=y
>>
>> then I added #define DEBUG  to both head.S and decompress.c under
>> arm/boot/compressed.
>> I added earlyprintk to the cmdline args used by the kexec bootloader but
>> nothing
>>
>> That kernel does boot silently, just 'Uncompresing..' then
>> 'XZ-compressed data is corrupt -- System halted'
>>
>> I do have output from the Angel bootloader at 9600 and by the kernel
>> at 115200 so the uart is working...
>> Any hint?
>
>
> If it is a problem with gcc (as Russell said before), there is no way you
> can fix it in the kernel. Only compiling it with an older gcc version can
> help.
>
> Maybe you can try to compile it for armv4 code.
>
> best regards,
> Petr

Hi,
yes, out of curtiosity I'll test on collie/sa1100 later.

As far as I can see, the disassembled code is indeed for armv5 (there
are 'bx lr') but apparently there are no misalignments.
I could only disassemble the gcc6 and gcc7 versions and upload for
competent eyes...

https://github.com/andrea-adami/gcc7-kernel-debug/tree/master/disassembled

Cheers
Andrea

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Dne 17.10.2017 v 10:18 Andrea Adami napsal(a):
> On Tue, Oct 17, 2017 at 10:10 AM, Petr Cvek <petrcvekcz@gmail.com> wrote:
>> Dne 17.10.2017 v 10:02 Andrea Adami napsal(a):
>>>
>>> <cut>
>>>
>>> Hi,
>>
>> Hi
>>
>>
>>>
>>> I am trying to debug further but I don't have any output on serial
>>> console...
>>> I enabled:
>>>
>>> CONFIG_DEBUG_LL=y
>>> CONFIG_DEBUG_PXA_UART1=y
>>> # CONFIG_DEBUG_ICEDCC is not set
>>> # CONFIG_DEBUG_SEMIHOSTING is not set
>>> # CONFIG_DEBUG_LL_UART_8250 is not set
>>> # CONFIG_DEBUG_LL_UART_PL01X is not set
>>> CONFIG_DEBUG_LL_INCLUDE="debug/8250.S"
>>> CONFIG_DEBUG_UART_8250=y
>>> CONFIG_DEBUG_UART_PHYS=0x40100000
>>> CONFIG_DEBUG_UART_VIRT=0xf6200000
>>> CONFIG_DEBUG_UART_8250_SHIFT=2
>>> # CONFIG_DEBUG_UART_8250_WORD is not set
>>> # CONFIG_DEBUG_UART_8250_PALMCHIP is not set
>>> # CONFIG_DEBUG_UART_8250_FLOW_CONTROL is not set
>>> CONFIG_UNCOMPRESS_INCLUDE="mach/uncompress.h"
>>> CONFIG_EARLY_PRINTK=y
>>>
>>> then I added #define DEBUG  to both head.S and decompress.c under
>>> arm/boot/compressed.
>>> I added earlyprintk to the cmdline args used by the kexec bootloader but
>>> nothing
>>>
>>> That kernel does boot silently, just 'Uncompresing..' then
>>> 'XZ-compressed data is corrupt -- System halted'
>>>
>>> I do have output from the Angel bootloader at 9600 and by the kernel
>>> at 115200 so the uart is working...
>>> Any hint?
>>
>>
>> If it is a problem with gcc (as Russell said before), there is no way you
>> can fix it in the kernel. Only compiling it with an older gcc version can
>> help.
>>

BTW you could add "alignment=2" (same as /proc/cpu/alignment) in your 
kernel parameters and kernel should check it even before you boot 
userspace.

But it will be *extra* slow as there will be many exceptions in the 
kernel and an emulation of the unaligned instructions. It is not a real 
permanent fix.

It should be possible to use up to march=armv5t as I elaborated it in 
the other mail.

best regards,
Petr

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Russell King - ARM Linux]

On Wed, Oct 18, 2017 at 12:39:13PM +0200, Petr Cvek wrote:
> 
> 
> Dne 17.10.2017 v 10:18 Andrea Adami napsal(a):
> >On Tue, Oct 17, 2017 at 10:10 AM, Petr Cvek <petrcvekcz@gmail.com> wrote:
> >>If it is a problem with gcc (as Russell said before), there is no way you
> >>can fix it in the kernel. Only compiling it with an older gcc version can
> >>help.
> >>
> 
> BTW you could add "alignment=2" (same as /proc/cpu/alignment) in your kernel
> parameters and kernel should check it even before you boot userspace.
> 
> But it will be *extra* slow as there will be many exceptions in the kernel
> and an emulation of the unaligned instructions. It is not a real permanent
> fix.

That's wrong.

Alignment exceptions are always fixed up in the kernel.  The parameter
(and what you set by writing /proc/cpu/alignment) sets the behaviour of
_userspace_ alignment exceptions.

> It should be possible to use up to march=armv5t as I elaborated it in the
> other mail.

If you're building for PXA, it's an ARMv5 architecture.  The kernel will
use -march=armv5te for all ARMv5 architectures.  Moreover, because one
of the PXA2* symbols should also be selected, CPU_XSCALE will also be
enabled, which means we will pass -mtune=xscale to the compiler.

The exception is if CPU_FEROCEON is enabled, in which case we will pass
-mtune=marvell-f to the kernel if supported by the compiler.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Dne 18.10.2017 v 12:44 Russell King - ARM Linux napsal(a):
> On Wed, Oct 18, 2017 at 12:39:13PM +0200, Petr Cvek wrote:
>>
>>
>> Dne 17.10.2017 v 10:18 Andrea Adami napsal(a):
>>> On Tue, Oct 17, 2017 at 10:10 AM, Petr Cvek <petrcvekcz@gmail.com> wrote:
>>>> If it is a problem with gcc (as Russell said before), there is no way you
>>>> can fix it in the kernel. Only compiling it with an older gcc version can
>>>> help.
>>>>
>>
>> BTW you could add "alignment=2" (same as /proc/cpu/alignment) in your kernel
>> parameters and kernel should check it even before you boot userspace.
>>
>> But it will be *extra* slow as there will be many exceptions in the kernel
>> and an emulation of the unaligned instructions. It is not a real permanent
>> fix.
> 
> That's wrong.
> 
> Alignment exceptions are always fixed up in the kernel.  The parameter
> (and what you set by writing /proc/cpu/alignment) sets the behaviour of
> _userspace_ alignment exceptions.

Oh sorry I didn't realize the problem is in the kernel decompressor. I 
thought it fails later in the kernel (for me first code which freezed 
was init).

> 
>> It should be possible to use up to march=armv5t as I elaborated it in the
>> other mail.
> 
> If you're building for PXA, it's an ARMv5 architecture.  The kernel will
> use -march=armv5te for all ARMv5 architectures.  Moreover, because one
> of the PXA2* symbols should also be selected, CPU_XSCALE will also be
> enabled, which means we will pass -mtune=xscale to the compiler.
> 
> The exception is if CPU_FEROCEON is enabled, in which case we will pass
> -mtune=marvell-f to the kernel if supported by the compiler.
> 

Yeah I know that, but forcing an older arch version would help as the 
7.1.0 code for armv5t is probably faster than armv4 (and certainly 
faster than gcc 6.3.0). Moreover as I tested armv5te/xscale (in 7.1.0) 
doesn't even generate the STRD bug (but it is slower than armv5t).

BTW if you compile kernel with 6.x (my testcase), then if you compile 
the userspace with armv5t (7.x) it will be faster. If you compile the 
userspace with armv5te (7.x) you still need the kernel parameter, 
because an unaligned access may corrupt init (possibly my testcase).

Petr

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

Hi,

good news!

The patch found in the other thread [1] does fix the issue.
Kernel 4.13 is booting now on corgi: I'll test 4.4 as countercheck.

Regards
Andrea


[1]
https://patchwork.kernel.org/patch/9944519/

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Andrea Adami]

On Tue, Oct 24, 2017 at 10:34 AM, Andrea Adami <andrea.adami@gmail.com> wrote:
> Hi,
>
> good news!
>
> The patch found in the other thread [1] does fix the issue.
> Kernel 4.13 is booting now on corgi: I'll test 4.4 as countercheck.
>
> Regards
> Andrea
>
>
> [1]
> https://patchwork.kernel.org/patch/9944519/


Maybe it is unclear, it was a gcc7 bug finally,

https://github.com/gcc-mirror/gcc/commit/f59996b56aaa1c1d62a16cbb4010775b624cbde0


No other patches have been applied.

Regards
Andrea

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Aaro Koskinen]

Hi,

On Mon, Oct 16, 2017 at 01:55:18PM +0200, Petr Cvek wrote:
> Dne 15.10.2017 v 12:46 Aaro Koskinen napsal(a):
> > This seems to be simpler to debug/reproduce using busybox. Compiling
> > just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
> > produces failing xz decompression:
> 
> Does it fail too when compiled with gcc 7.2 and with march=armv5t and/or
> march=armv4t ?

At least armv4t seems to work.

> > root at thecus-n2100:~$ gzip foo.txt
> > root at thecus-n2100:~$ ./busybox.gcc-7.2 gzip -d foo.txt.gz
> > gzip: crc error
> > root at thecus-n2100:~$ ./busybox.gcc-6.4 gzip -d foo.txt.gz
> > root at thecus-n2100:~$ cat foo.txt
> > foo
> > 
> 
> Is it really compiled with armv5te ("readelf -n -A")?

Yes.

> I've just found my old gcc 6.3 compiled some code with a default armv4t.

You can set the default with compiling GCC using --with-arch=...

BTW, the failing instruction seems to be STRD and there seems to be
already a related bug report:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82445

A.

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Dne 17.10.2017 v 22:33 Aaro Koskinen napsal(a):
> Hi,
> 
Hi,

> On Mon, Oct 16, 2017 at 01:55:18PM +0200, Petr Cvek wrote:
>> Dne 15.10.2017 v 12:46 Aaro Koskinen napsal(a):
>>> This seems to be simpler to debug/reproduce using busybox. Compiling
>>> just busybox with GCC 7.2 and march=armv5te/mtune=xscale already
>>> produces failing xz decompression:
>>
>> Does it fail too when compiled with gcc 7.2 and with march=armv5t and/or
>> march=armv4t ?
> 
> At least armv4t seems to work.
> 

Are you sure you used gcc 7.2 and parameters march=armv5te + 
mtune=xscale? I've compiled the bugzilla code with these same parameters:

	-march=armv5te -mtune=xscale -marm -O2  -c strd.c

For me (on gcc 7.1.0) only mtune=xscale/iwmmxt doesn't generate STRD. 
Other mtune variants (arm1020e, arm926ej-s generate it). Here is a 
table, which was tested on a real hardware.

-march=armv5te -mtune=xscale	OK, code very long
-march=armv5te -mtune=iwmmxt	OK, same code as v5te/xscale
-march=iwmmxt			OK, same code as v5te/xscale
-march=armv5t			OK, less opcodes than v5te/xscale
-march=armv5			OK, same as armv5t
-march=armv5te			KO, unaligned STRD, alignment exception
-march=armv5tej			KO, unaligned STRD (PXA27x is only v5te)

BTW gcc 6.3.0 generates even longer code and it doesn't support armvtej. 
Perhaps your 7.2 has even more optimalizations even for mtune=xscale.

So I suppose compiling kernel and userspace with -match=armv5t or less 
should be fine for now (unless there is another similar problem due to 
7.x optimizations).

> 
>> I've just found my old gcc 6.3 compiled some code with a default armv4t.
> 
> You can set the default with compiling GCC using --with-arch=...

Yeah I know about that, it seems something probably override my settings 
(most likely my scripts for separate crosstool and buildroot).

> 
> BTW, the failing instruction seems to be STRD and there seems to be
> already a related bug report:
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82445

Big thanks for info.

> 
> A.
> 

Petr

ARM: pxa/corgi: armv5te kernel 4.12 fails to decompress compiled with gcc7

[Petr Cvek]

Hi,
I wasn't compiling GCC takes me few hours and I have no idea where to 
start (don't have JTAG on my PXA machine, and gdb does show anything 
useful). I wanted to look at that in the summer, but I've had other work.

I suggest you to compile kernel with an older GCC. Maybe even the whole 
OS (busybox) with a working one but it seems that the failures in 
standalone application have a really low occurences of bugs. It is 
possible the bug is somewhere in libc etc.

Petr

Dne 14.10.2017 v 23:50 Aaro Koskinen napsal(a):
> Hi,
> 
> On Sat, Sep 02, 2017 at 11:50:55PM +0200, Andrea Adami wrote:
>> On Thu, Jul 20, 2017 at 8:57 AM, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
>>> Andrea Adami <andrea.adami@gmail.com> writes:
>>> I have the same report on userspace side on buildroot from Petr [1], which
>>> triggers endless segfaults in userspace (init) with gcc7 while everything is
>>> fine with gcc6. I have confirmation on my test farm the problem happens as well.
>>>
>>> Since debugging in userspace is far easier, I would suggest attacking the debug
>>> with Petr on userspace side, and once sorted out, come back to kernel side.
>>
>> sorry for the delay, I was awaiting to test gcc 7.2 and binutils 2.29.
>> Unfortunately, same issue: XZ-compressed data is corrupt - System Halted.
> 
> Has anyone been able to debug this further? It seems there are also issues
> on iop32x/n2100 with GCC 7.2 and binutils 2.29.1 - early boot hangs silently
> and I cannot get earlyprintk to work. Compiling with GCC 6.4.0 it's OK.
> 
> A.
>