Re: Kernel crosscompilers

Re: Kernel crosscompilers

[Christophe Leroy]

Le 06/05/2021 à 16:17, Arnd Bergmann a écrit :
> On Thu, May 6, 2021 at 2:42 PM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>>
>> Hello Arnd,
>>
>> May I ask you whether you plan to build cross compilers based on GCC 11.1 at
>> https://mirrors.edge.kernel.org/pub/tools/crosstool/ ?
> 
> Hi Christophe,
> 
> I've built a snapshot a few days before the release, that one is
> identical to 11.1
> except for the reported version number. I've tried to ask around for
> help testing
> this, but so far I have not heard from anyone.
> 
> Building a new set of compilers takes around a day on my build box, so I want
> to make sure I don't have to do it more often than necessary. If you are able
> to give the binaries a spin, preferably on a ppc64le or arm64 host, please let
> me know how it goes and I'll rebuilt them on the release tag.
> 

Hi Arnd,

I don't have any ppc or arm host I can build on.
I'm building on x86 for powerpc embedded boards.

I have tried your GCC 11 snapshot, I get something booting but it crashes when launching init.

[    7.368410] init[1]: bad frame in sys_sigreturn: 7fb2fd60 nip 001083cc lr 001083c4
[    7.376283] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    7.383680] CPU: 0 PID: 1 Comm: init Not tainted 5.12.0-s3k-dev-16316-g9e799d5df185 #5054
[    7.391767] Call Trace:
[    7.394174] [c9023db0] [c00211e8] panic+0x130/0x304 (unreliable)
[    7.400112] [c9023e10] [c0024e68] do_exit+0x874/0x910
[    7.405104] [c9023e50] [c0024f80] do_group_exit+0x40/0xc4
[    7.410440] [c9023e60] [c0033334] get_signal+0x1d8/0x93c
[    7.415689] [c9023ec0] [c0007f34] do_notify_resume+0x6c/0x314
[    7.421369] [c9023f20] [c000d580] syscall_exit_prepare+0x120/0x184
[    7.427479] [c9023f30] [c001101c] ret_from_syscall+0xc/0x28

Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this 
cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.

Seems like there is something wrong with it with GCC11. When forcing CONFIG_CC_HAS_ASM_GOTO_OUTPUT 
to 'n', the kernel boots ok.

Christophe

Re: Kernel crosscompilers

[Christophe Leroy]

Le 08/05/2021 à 08:46, Christophe Leroy a écrit :
> 
> 
> Le 06/05/2021 à 16:17, Arnd Bergmann a écrit :
>> On Thu, May 6, 2021 at 2:42 PM Christophe Leroy
>> <christophe.leroy@csgroup.eu> wrote:
>>>
>>> Hello Arnd,
>>>
>>> May I ask you whether you plan to build cross compilers based on GCC 11.1 at
>>> https://mirrors.edge.kernel.org/pub/tools/crosstool/ ?
>>
>> Hi Christophe,
>>
>> I've built a snapshot a few days before the release, that one is
>> identical to 11.1
>> except for the reported version number. I've tried to ask around for
>> help testing
>> this, but so far I have not heard from anyone.
>>
>> Building a new set of compilers takes around a day on my build box, so I want
>> to make sure I don't have to do it more often than necessary. If you are able
>> to give the binaries a spin, preferably on a ppc64le or arm64 host, please let
>> me know how it goes and I'll rebuilt them on the release tag.
>>
> 
> Hi Arnd,
> 
> I don't have any ppc or arm host I can build on.
> I'm building on x86 for powerpc embedded boards.
> 
> I have tried your GCC 11 snapshot, I get something booting but it crashes when launching init.
> 
> [    7.368410] init[1]: bad frame in sys_sigreturn: 7fb2fd60 nip 001083cc lr 001083c4
> [    7.376283] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [    7.383680] CPU: 0 PID: 1 Comm: init Not tainted 5.12.0-s3k-dev-16316-g9e799d5df185 #5054
> [    7.391767] Call Trace:
> [    7.394174] [c9023db0] [c00211e8] panic+0x130/0x304 (unreliable)
> [    7.400112] [c9023e10] [c0024e68] do_exit+0x874/0x910
> [    7.405104] [c9023e50] [c0024f80] do_group_exit+0x40/0xc4
> [    7.410440] [c9023e60] [c0033334] get_signal+0x1d8/0x93c
> [    7.415689] [c9023ec0] [c0007f34] do_notify_resume+0x6c/0x314
> [    7.421369] [c9023f20] [c000d580] syscall_exit_prepare+0x120/0x184
> [    7.427479] [c9023f30] [c001101c] ret_from_syscall+0xc/0x28
> 
> Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this 
> cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.
> 
> Seems like there is something wrong with it with GCC11. When forcing CONFIG_CC_HAS_ASM_GOTO_OUTPUT 
> to 'n', the kernel boots ok.
> 

I found the problem, that's due to r10 register being reused by GCC in the copy loop below:

   10:	7d 09 03 a6 	mtctr   r8
   14:	80 ca 00 00 	lwz     r6,0(r10)
   18:	80 ea 00 04 	lwz     r7,4(r10)
   1c:	90 c9 00 08 	stw     r6,8(r9)
   20:	90 e9 00 0c 	stw     r7,12(r9)
   24:	39 0a 00 08 	addi    r8,r10,8
   28:	39 29 00 08 	addi    r9,r9,8
=>2c:	81 4a 00 08 	lwz     r10,8(r10)
   30:	81 6a 00 0c 	lwz     r11,12(r10)
   34:	91 49 00 08 	stw     r10,8(r9)
   38:	91 69 00 0c 	stw     r11,12(r9)
   3c:	39 48 00 08 	addi    r10,r8,8
   40:	39 29 00 08 	addi    r9,r9,8
   44:	42 00 ff d0 	bdnz    14 <__unsafe_restore_general_regs+0x14>

earlyclobber modifier is missing in the CONFIG_CC_HAS_ASM_GOTO_OUTPUT version of __get_user_asm2_goto().

This is not a GCC bug, with that fixed my board is working OK.

Christophe

Re: Kernel crosscompilers

[Arnd Bergmann]

On Sat, May 8, 2021 at 11:22 AM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
> Le 08/05/2021 à 08:46, Christophe Leroy a écrit :
> >
> > I don't have any ppc or arm host I can build on.
> > I'm building on x86 for powerpc embedded boards.
> >
> > I have tried your GCC 11 snapshot, I get something booting but it crashes when launching init.
> >
...
> > Seems like there is something wrong with it with GCC11. When forcing CONFIG_CC_HAS_ASM_GOTO_OUTPUT
> > to 'n', the kernel boots ok.
> >
>
> I found the problem, that's due to r10 register being reused by GCC in the copy loop below:

Ok, thanks for testing. I'll try to rebuild the release tag in the
next few days.

      Arnd

Re: Kernel crosscompilers

[Arnd Bergmann]

On Sat, May 8, 2021 at 8:46 AM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
> Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this
> cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.

BTW, can you point me to those patches? I think it would be nice if we
could eventually
converge parts of get_user()/put_user() implementation on something
that works for all
architectures, we do seem to rewrite these way too often. Ideally we'd
have something
in asm-generic that provides all the wrappers, and just requires an
architecture to
implement the inline asm helpers for each of the sizes.

      Arnd

Re: Kernel crosscompilers

[Christophe Leroy]

Le 08/05/2021 à 11:43, Arnd Bergmann a écrit :
> On Sat, May 8, 2021 at 8:46 AM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>>
>> Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this
>> cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.
> 
> BTW, can you point me to those patches? I think it would be nice if we
> could eventually
> converge parts of get_user()/put_user() implementation on something
> that works for all
> architectures, we do seem to rewrite these way too often. Ideally we'd
> have something
> in asm-generic that provides all the wrappers, and just requires an
> architecture to
> implement the inline asm helpers for each of the sizes.
> 

The series that is merged for 5.13 is here: 
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=233250&state=*

And of course the result is just 
https://github.com/linuxppc/linux/blob/master/arch/powerpc/include/asm/uaccess.h
and all its history at 
https://github.com/linuxppc/linux/commits/master/arch/powerpc/include/asm/uaccess.h

The fix from today here: 
https://patchwork.ozlabs.org/project/linuxppc-dev/patch/cf0a050d124d4f426cdc7a74009d17b01d8d8969.1620465917.git.christophe.leroy@csgroup.eu/

I made a try to use asm-generic/uaccess.h a few weeks ago, but it was not conclusive. I can't 
remember the details, but I can have a new look at it next week if you are interested with the details.

Christophe

Re: Kernel crosscompilers

[Arnd Bergmann]

On Sat, May 8, 2021 at 12:15 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
> Le 08/05/2021 à 11:43, Arnd Bergmann a écrit :
> > On Sat, May 8, 2021 at 8:46 AM Christophe Leroy
> > <christophe.leroy@csgroup.eu> wrote:
> >>
> >> Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this
> >> cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.
> >
> > BTW, can you point me to those patches? I think it would be nice if we
> > could eventually
> > converge parts of get_user()/put_user() implementation on something
> > that works for all
> > architectures, we do seem to rewrite these way too often. Ideally we'd
> > have something
> > in asm-generic that provides all the wrappers, and just requires an
> > architecture to
> > implement the inline asm helpers for each of the sizes.
> >
>
> The series that is merged for 5.13 is here:
> https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=233250&state=*
>
> And of course the result is just
> https://github.com/linuxppc/linux/blob/master/arch/powerpc/include/asm/uaccess.h
> and all its history at
> https://github.com/linuxppc/linux/commits/master/arch/powerpc/include/asm/uaccess.h

Ah, I should have looked there first ;-)

> The fix from today here:
> https://patchwork.ozlabs.org/project/linuxppc-dev/patch/cf0a050d124d4f426cdc7a74009d17b01d8d8969.1620465917.git.christophe.leroy@csgroup.eu/
>
> I made a try to use asm-generic/uaccess.h a few weeks ago, but it was not conclusive. I can't
> remember the details, but I can have a new look at it next week if you are interested with the details.

I hope you didn't waste too much time on that. The current asm-generic/uaccess.h
code is not helpful anywhere and needs to be rewritten before we can have
normal architectures using it.

        Arnd

Re: Kernel crosscompilers

[Michael Ellerman]

Christophe Leroy <christophe.leroy@csgroup.eu> writes:
> Le 08/05/2021 à 08:46, Christophe Leroy a écrit :
>> Le 06/05/2021 à 16:17, Arnd Bergmann a écrit :
>>> On Thu, May 6, 2021 at 2:42 PM Christophe Leroy
>>> <christophe.leroy@csgroup.eu> wrote:
>>>>
>>>> Hello Arnd,
>>>>
>>>> May I ask you whether you plan to build cross compilers based on GCC 11.1 at
>>>> https://mirrors.edge.kernel.org/pub/tools/crosstool/ ?
>>>
>>> Hi Christophe,
>>>
>>> I've built a snapshot a few days before the release, that one is
>>> identical to 11.1
>>> except for the reported version number. I've tried to ask around for
>>> help testing
>>> this, but so far I have not heard from anyone.
>>>
>>> Building a new set of compilers takes around a day on my build box, so I want
>>> to make sure I don't have to do it more often than necessary. If you are able
>>> to give the binaries a spin, preferably on a ppc64le or arm64 host, please let
>>> me know how it goes and I'll rebuilt them on the release tag.
>>>
>> 
>> Hi Arnd,
>> 
>> I don't have any ppc or arm host I can build on.
>> I'm building on x86 for powerpc embedded boards.
>> 
>> I have tried your GCC 11 snapshot, I get something booting but it crashes when launching init.
>> 
>> [    7.368410] init[1]: bad frame in sys_sigreturn: 7fb2fd60 nip 001083cc lr 001083c4
>> [    7.376283] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>> [    7.383680] CPU: 0 PID: 1 Comm: init Not tainted 5.12.0-s3k-dev-16316-g9e799d5df185 #5054
>> [    7.391767] Call Trace:
>> [    7.394174] [c9023db0] [c00211e8] panic+0x130/0x304 (unreliable)
>> [    7.400112] [c9023e10] [c0024e68] do_exit+0x874/0x910
>> [    7.405104] [c9023e50] [c0024f80] do_group_exit+0x40/0xc4
>> [    7.410440] [c9023e60] [c0033334] get_signal+0x1d8/0x93c
>> [    7.415689] [c9023ec0] [c0007f34] do_notify_resume+0x6c/0x314
>> [    7.421369] [c9023f20] [c000d580] syscall_exit_prepare+0x120/0x184
>> [    7.427479] [c9023f30] [c001101c] ret_from_syscall+0xc/0x28
>> 
>> Something is going wrong with asm goto output. I implemented get_user() helpers with asm goto this 
>> cycle (commit 5cd29b1fd3e8). I tested it with CLANG before submitting, it was working.
>> 
>> Seems like there is something wrong with it with GCC11. When forcing CONFIG_CC_HAS_ASM_GOTO_OUTPUT 
>> to 'n', the kernel boots ok.
>> 
>
> I found the problem, that's due to r10 register being reused by GCC in the copy loop below:
>
>    10:	7d 09 03 a6 	mtctr   r8
>    14:	80 ca 00 00 	lwz     r6,0(r10)
>    18:	80 ea 00 04 	lwz     r7,4(r10)
>    1c:	90 c9 00 08 	stw     r6,8(r9)
>    20:	90 e9 00 0c 	stw     r7,12(r9)
>    24:	39 0a 00 08 	addi    r8,r10,8
>    28:	39 29 00 08 	addi    r9,r9,8
> =>2c:	81 4a 00 08 	lwz     r10,8(r10)
>    30:	81 6a 00 0c 	lwz     r11,12(r10)
>    34:	91 49 00 08 	stw     r10,8(r9)
>    38:	91 69 00 0c 	stw     r11,12(r9)
>    3c:	39 48 00 08 	addi    r10,r8,8
>    40:	39 29 00 08 	addi    r9,r9,8
>    44:	42 00 ff d0 	bdnz    14 <__unsafe_restore_general_regs+0x14>
>
> earlyclobber modifier is missing in the CONFIG_CC_HAS_ASM_GOTO_OUTPUT version of __get_user_asm2_goto().

Thanks for tracking that down. I hit it last week when testing Arnd's
compilers but hadn't had time to find the root cause.

cheers