help with clang build for mips

help with clang build for mips

[Sudip Mukherjee]

Hi All,

I am trying to build the kernel for mips architecture with clang. I
see some of the configuration builds properly, but some of the configs
fails with the error (fuloong2e_defconfig):

error: unknown target CPU 'loongson2e'
note: valid target CPU values are: mips1, mips2, mips3, mips4, mips5,
mips32, mips32r2, mips32r3, mips32r5, mips32r6, mips64, mips64r2,
mips64r3, mips64r5, mips64r6, octeon, octeon+, p5600

Another one fails with (jazz_defconfig):
error: unknown target CPU 'r4600'

is there any way to build these configs which fails with "unknown
target CPU". or they should be skipped for a clang build?


-- 
Regards
Sudip

Re: help with clang build for mips

[Nathan Chancellor]

Hi Sudip,

On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> Hi All,
> 
> I am trying to build the kernel for mips architecture with clang. I
> see some of the configuration builds properly, but some of the configs
> fails with the error (fuloong2e_defconfig):
> 
> error: unknown target CPU 'loongson2e'
> note: valid target CPU values are: mips1, mips2, mips3, mips4, mips5,
> mips32, mips32r2, mips32r3, mips32r5, mips32r6, mips64, mips64r2,
> mips64r3, mips64r5, mips64r6, octeon, octeon+, p5600
> 
> Another one fails with (jazz_defconfig):
> error: unknown target CPU 'r4600'
> 
> is there any way to build these configs which fails with "unknown
> target CPU". or they should be skipped for a clang build?

I suppose it comes down to how critical the '-march' value is to getting
a working kernel. In other words, can we ditch the current '-march'
values for something that is supported by both GCC and LLVM without
regressing GCC? I am not sure, that probably requires input from the
MIPS maintainers. This isn't the only road block though, there are
certain compiler options that are needed for the kernels to run due to
errata, which are not implemented in LLVM:

https://github.com/ClangBuiltLinux/linux/issues/1544

Unfortunately, I doubt there is much interest in that work, given MIPS
as an architecture isn't being developed anymore. Might be an
interesting hobbyist project though. It is probably best to just test
configurations that work and make sure that those do not regress. Others
are more than welcome to weigh in though!

Cheers,
Nathan

Re: help with clang build for mips

[Arnd Bergmann]

On Thu, Jul 14, 2022 at 7:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
>
> Unfortunately, I doubt there is much interest in that work, given MIPS
> as an architecture isn't being developed anymore. Might be an
> interesting hobbyist project though. It is probably best to just test
> configurations that work and make sure that those do not regress. Others
> are more than welcome to weigh in though!

I think there is still enough interest in the newer loongson64 mips machines,
which should work with clang, but the loongson2e/f machines are way more
obscure now. Debian dropped support for these a couple of years ago.

        Arnd

Re: help with clang build for mips

[Sudip Mukherjee]

On Thu, Jul 14, 2022 at 9:11 PM Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Thu, Jul 14, 2022 at 7:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> >
> > Unfortunately, I doubt there is much interest in that work, given MIPS
> > as an architecture isn't being developed anymore. Might be an
> > interesting hobbyist project though. It is probably best to just test
> > configurations that work and make sure that those do not regress. Others
> > are more than welcome to weigh in though!
>
> I think there is still enough interest in the newer loongson64 mips machines,
> which should work with clang, but the loongson2e/f machines are way more
> obscure now. Debian dropped support for these a couple of years ago.

And, here is a complete list.

 unknown target CPU 'r4600'
 unknown target CPU 'r5000'
 unknown target CPU 'r4100'
 unknown target CPU 'r4600'
 unknown target CPU 'r5000'
 unknown target CPU 'r4100'
 unknown target CPU 'r3000'
 unknown target CPU 'r4100'
 unknown target CPU 'loongson2f'
 unknown target CPU 'r4100'
 unknown target CPU 'loongson2e'
 unknown target CPU 'r5000'
 unknown target CPU 'r4600'
 unknown target CPU 'r5000'
 unknown target CPU 'r8000'
 unknown target CPU 'r4100'

generated by:

rbtx49xx_defconfig
decstation_r4k_defconfig
ip32_defconfig
capcella_defconfig
rm200_defconfig
cobalt_defconfig
mpc30x_defconfig
workpad_defconfig
decstation_defconfig
tb0219_defconfig
lemote2f_defconfig
tb0287_defconfig
fuloong2e_defconfig
sb1250_swarm_defconfig
ip22_defconfig
jazz_defconfig
bigsur_defconfig
ip27_defconfig
e55_defconfig
tb0226_defconfig


-- 
Regards
Sudip

Re: help with clang build for mips

[Arnd Bergmann]

On Fri, Jul 15, 2022 at 11:59 AM Sudip Mukherjee
<sudipm.mukherjee@gmail.com> wrote:
>
> On Thu, Jul 14, 2022 at 9:11 PM Arnd Bergmann <arnd@kernel.org> wrote:
> >
> > On Thu, Jul 14, 2022 at 7:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > > On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> > >
> > > Unfortunately, I doubt there is much interest in that work, given MIPS
> > > as an architecture isn't being developed anymore. Might be an
> > > interesting hobbyist project though. It is probably best to just test
> > > configurations that work and make sure that those do not regress. Others
> > > are more than welcome to weigh in though!
> >
> > I think there is still enough interest in the newer loongson64 mips machines,
> > which should work with clang, but the loongson2e/f machines are way more
> > obscure now. Debian dropped support for these a couple of years ago.
>
> And, here is a complete list.
>
>  unknown target CPU 'r4600'
>  unknown target CPU 'r5000'
>  unknown target CPU 'r4100'
>  unknown target CPU 'r4600'
>  unknown target CPU 'r5000'
>  unknown target CPU 'r4100'
>  unknown target CPU 'r3000'
>  unknown target CPU 'r4100'
>  unknown target CPU 'loongson2f'
>  unknown target CPU 'r4100'
>  unknown target CPU 'loongson2e'
>  unknown target CPU 'r5000'
>  unknown target CPU 'r4600'
>  unknown target CPU 'r5000'
>  unknown target CPU 'r8000'
>  unknown target CPU 'r4100'

I wonder if we could just change the kernel Makefile to fall
back to mips2 (r3000), mips3 (r4x00, loongson2) and mips4
(r5000, r8000) respectively.

       Arnd

Re: help with clang build for mips

[Sudip Mukherjee]

On Thu, Jul 14, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Hi Sudip,
>
> On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> > Hi All,
> >
> > I am trying to build the kernel for mips architecture with clang. I
> > see some of the configuration builds properly, but some of the configs
> > fails with the error (fuloong2e_defconfig):
> >

<snip>

>
> Unfortunately, I doubt there is much interest in that work, given MIPS
> as an architecture isn't being developed anymore. Might be an
> interesting hobbyist project though. It is probably best to just test
> configurations that work and make sure that those do not regress. Others
> are more than welcome to weigh in though!

So, I have now disabled those builds with unsupported CPU. But I do
see some others with linux-next and will like to have your feedback on
them please.

mips clang cavium_octeon_defconfig:

clang-15: error: unsupported argument '-mfix-cn63xxp1' to option '-Wa,'
make[1]: *** [scripts/Makefile.build:250: scripts/mod/empty.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:1290: prepare0] Error 2

+++++++++++++++++++++++++++++++++++++

mips clang decstation_64_defconfig:

arch/mips/Makefile:281: *** CONFIG_CPU_DADDI_WORKAROUNDS unsupported
without -msym32.  Stop.

+++++++++++++++++++++++++++++++++++++

mips clang ip28_defconfig:

clang-15: error: unknown argument: '-mr10k-cache-barrier=store'
clang-15: error: unknown argument: '-mr10k-cache-barrier=store'
make[1]: *** [scripts/Makefile.build:118: kernel/bounds.s] Error 1
make: *** [arch/mips/Makefile:407: archprepare] Error 2

+++++++++++++++++++++++++++++++++++++

mips clang loongson2k_defconfig:

error: ABI 'o32' is not supported on CPU 'mips64r2'
make[1]: *** [Kbuild:48: missing-syscalls] Error 1
make: *** [arch/mips/Makefile:409: archprepare] Error 2

+++++++++++++++++++++++++++++++++++++

mips clang loongson3_defconfig:

error: ABI 'o32' is not supported on CPU 'mips64r2'
make[1]: *** [Kbuild:48: missing-syscalls] Error 1
make: *** [arch/mips/Makefile:409: archprepare] Error 2

+++++++++++++++++++++++++++++++++++++



-- 
Regards
Sudip

Re: help with clang build for mips

[Nathan Chancellor]

Hi Sudip,

On Fri, Jul 22, 2022 at 02:29:01PM +0100, Sudip Mukherjee wrote:
> On Thu, Jul 14, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > Hi Sudip,
> >
> > On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> > > Hi All,
> > >
> > > I am trying to build the kernel for mips architecture with clang. I
> > > see some of the configuration builds properly, but some of the configs
> > > fails with the error (fuloong2e_defconfig):
> > >
> 
> <snip>
> 
> >
> > Unfortunately, I doubt there is much interest in that work, given MIPS
> > as an architecture isn't being developed anymore. Might be an
> > interesting hobbyist project though. It is probably best to just test
> > configurations that work and make sure that those do not regress. Others
> > are more than welcome to weigh in though!
> 
> So, I have now disabled those builds with unsupported CPU. But I do
> see some others with linux-next and will like to have your feedback on
> them please.

Thanks for testing!

> mips clang cavium_octeon_defconfig:
> 
> clang-15: error: unsupported argument '-mfix-cn63xxp1' to option '-Wa,'
> make[1]: *** [scripts/Makefile.build:250: scripts/mod/empty.o] Error 1
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:1290: prepare0] Error 2
> 
> +++++++++++++++++++++++++++++++++++++
> 
> mips clang decstation_64_defconfig:
> 
> arch/mips/Makefile:281: *** CONFIG_CPU_DADDI_WORKAROUNDS unsupported
> without -msym32.  Stop.
> 
> +++++++++++++++++++++++++++++++++++++
> 
> mips clang ip28_defconfig:
> 
> clang-15: error: unknown argument: '-mr10k-cache-barrier=store'
> clang-15: error: unknown argument: '-mr10k-cache-barrier=store'
> make[1]: *** [scripts/Makefile.build:118: kernel/bounds.s] Error 1
> make: *** [arch/mips/Makefile:407: archprepare] Error 2

It sounds like these three are all just unsupported flags to the
compiler or assembler:

https://github.com/ClangBuiltLinux/linux/issues/1544

These all appear to be workaround for errata so I am not sure they
should just be wrapped in cc-option; I don't think there is much point
to building kernels that don't have the proper workarounds to actually
run on real hardware.

It looks like the -msym32 one can be worked around with 'KBUILD_SYM32=n'
in your make command though?

> +++++++++++++++++++++++++++++++++++++
> 
> mips clang loongson2k_defconfig:
> 
> error: ABI 'o32' is not supported on CPU 'mips64r2'
> make[1]: *** [Kbuild:48: missing-syscalls] Error 1
> make: *** [arch/mips/Makefile:409: archprepare] Error 2
> 
> +++++++++++++++++++++++++++++++++++++
> 
> mips clang loongson3_defconfig:
> 
> error: ABI 'o32' is not supported on CPU 'mips64r2'
> make[1]: *** [Kbuild:48: missing-syscalls] Error 1
> make: *** [arch/mips/Makefile:409: archprepare] Error 2
> 
> +++++++++++++++++++++++++++++++++++++

The o32 ABI is not supported by LLVM and it sounds like it is unlikely
to without someone stepping up:

https://github.com/ClangBuiltLinux/linux/issues/884

https://llvm.org/pr38063

There was a patch to disable this on the kernel side but it was never
picked up due to what I read as a soft NAK:

https://lore.kernel.org/1614838778-3696-1-git-send-email-yangtiezhu@loongson.cn/

https://lore.kernel.org/alpine.DEB.2.21.2103042356150.51127@angie.orcam.me.uk/

Cheers,
Nathan

Re: help with clang build for mips

[Sudip Mukherjee]

On Fri, Jul 22, 2022 at 4:21 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Hi Sudip,
>
> On Fri, Jul 22, 2022 at 02:29:01PM +0100, Sudip Mukherjee wrote:
> > On Thu, Jul 14, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > Hi Sudip,
> > >
> > > On Thu, Jul 14, 2022 at 05:39:39PM +0100, Sudip Mukherjee wrote:
> > > > Hi All,
> > > >
> > > > I am trying to build the kernel for mips architecture with clang. I
> > > > see some of the configuration builds properly, but some of the configs
> > > > fails with the error (fuloong2e_defconfig):
> > > >
> >
> > <snip>
> >
> > >
> > > Unfortunately, I doubt there is much interest in that work, given MIPS
> > > as an architecture isn't being developed anymore. Might be an
> > > interesting hobbyist project though. It is probably best to just test
> > > configurations that work and make sure that those do not regress. Others
> > > are more than welcome to weigh in though!
> >
> > So, I have now disabled those builds with unsupported CPU. But I do
> > see some others with linux-next and will like to have your feedback on
> > them please.
>
> Thanks for testing!
>

<snip>

>
> There was a patch to disable this on the kernel side but it was never
> picked up due to what I read as a soft NAK:
>

Thanks Nathan. I have now disabled all these configs in my daily builds.


-- 
Regards
Sudip

Re: help with clang build for mips

[Arnd Bergmann]

On Fri, Jul 22, 2022 at 5:21 PM Nathan Chancellor <nathan@kernel.org> wrote:
> On Fri, Jul 22, 2022 at 02:29:01PM +0100, Sudip Mukherjee wrote:
> > On Thu, Jul 14, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> The o32 ABI is not supported by LLVM and it sounds like it is unlikely
> to without someone stepping up:
>
> https://github.com/ClangBuiltLinux/linux/issues/884
>
> https://llvm.org/pr38063
>
> There was a patch to disable this on the kernel side but it was never
> picked up due to what I read as a soft NAK:
>
> https://lore.kernel.org/1614838778-3696-1-git-send-email-yangtiezhu@loongson.cn/
>
> https://lore.kernel.org/alpine.DEB.2.21.2103042356150.51127@angie.orcam.me.uk/

I think we can just give up on the combination of clang and mips in
that case, it's
just not worth it. As Maciej said, not supporting 32-bit mode on any modern CPU
is a joke, as most of the remaining mips machines are 32-bit-only, and
many 64-bit
machines run 32-bit code because of memory constraints.

         Arnd

Re: help with clang build for mips

[Nathan Chancellor]

On Sun, Jul 24, 2022 at 04:49:56PM +0200, Arnd Bergmann wrote:
> On Fri, Jul 22, 2022 at 5:21 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > On Fri, Jul 22, 2022 at 02:29:01PM +0100, Sudip Mukherjee wrote:
> > > On Thu, Jul 14, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > The o32 ABI is not supported by LLVM and it sounds like it is unlikely
> > to without someone stepping up:
> >
> > https://github.com/ClangBuiltLinux/linux/issues/884
> >
> > https://llvm.org/pr38063
> >
> > There was a patch to disable this on the kernel side but it was never
> > picked up due to what I read as a soft NAK:
> >
> > https://lore.kernel.org/1614838778-3696-1-git-send-email-yangtiezhu@loongson.cn/
> >
> > https://lore.kernel.org/alpine.DEB.2.21.2103042356150.51127@angie.orcam.me.uk/
> 
> I think we can just give up on the combination of clang and mips in
> that case, it's
> just not worth it. As Maciej said, not supporting 32-bit mode on any modern CPU
> is a joke, as most of the remaining mips machines are 32-bit-only, and
> many 64-bit
> machines run 32-bit code because of memory constraints.

Doesn't this configuration only impact 64-bit kernels running 32-bit
binaries, rather than 64-bit chips running 32-bit kernels (which seems
to be what your last sentence is concerned with, if I understand you
correctly)? We can still build and boot some 32-bit MIPS configurations
in QEMU and I believe Paul Cercueil has clang built kernels running on
real hardware.

Don't get me wrong, of all the architectures that we claim to support,
MIPS definitely has the weakest support in LLVM, as it is no longer
being actively developed from what I can see. The couple of crashes I
have reported have gotten fixed pretty quickly but that is only because
they were relatively simple. For the more complicated issues, I doubt we
would have an easy time getting those fixed without more support. Until
that happens (if it ever does), it probably isn't worth testing MIPS in
any sort of continuous capacity aside from the few configurations we
have working already (I test malta_defconfig + the generic 32r*
configs).

Cheers,
Nathan

Re: help with clang build for mips

[Arnd Bergmann]

On Mon, Jul 25, 2022 at 6:02 PM Nathan Chancellor <nathan@kernel.org> wrote:
> On Sun, Jul 24, 2022 at 04:49:56PM +0200, Arnd Bergmann wrote:
> >
> > I think we can just give up on the combination of clang and mips in
> > that case, it's
> > just not worth it. As Maciej said, not supporting 32-bit mode on any modern CPU
> > is a joke, as most of the remaining mips machines are 32-bit-only, and
> > many 64-bit
> > machines run 32-bit code because of memory constraints.
>
> Doesn't this configuration only impact 64-bit kernels running 32-bit
> binaries, rather than 64-bit chips running 32-bit kernels (which seems
> to be what your last sentence is concerned with, if I understand you
> correctly)? We can still build and boot some 32-bit MIPS configurations
> in QEMU and I believe Paul Cercueil has clang built kernels running on
> real hardware.

It's the same thing, building 32-bit kernels for 64-bit CPUs fails just
as well:

$ clang --target=mips64-linux -xc /dev/null -o /dev/null  -E
-march=octeon -m32 -mabi=o32
error: ABI 'o32' is not supported on CPU 'octeon'

And it's not like this is tied to code generation at all, the script that
does this just needs the preprocessor macros for a 32-bit environment
in order to generate the compat system call table in the 64-bit kernel.

We could in theory work around this by replacing the -march= flags
with a lookup table that translates each 64-bit CPU to the closest
32-bit CPU that is a direct subset, but this is something the kernel
should not have to do when the clang driver can just do it.

In gcc, these two command lines do the same thing:

$ mips64-linux-gcc -march=mips32r2
$ mips64-linux-gcc -march=mips64r2 -mabi=32

Whereas in clang, one works as expected and the other one throws
an error. The explanation given in the bug report was that it would
be too much work to make llvm generate object code that uses the
64-bit-only instructions but the 32-bit ABI, but this is not what we
need, or what gcc does, we just want the command line to be
interpreted sanely to give 32-bit output, just as it does on x86, powerpc
and s390 when using the -m32 flag to produce 32-bit code.

Note that a compiler that produces object code with 64-bit instructions
but the 32-bit ABI would not even work on 32-bit kernels because the
upper halves of the registers are not saved across context switches.

> Don't get me wrong, of all the architectures that we claim to support,
> MIPS definitely has the weakest support in LLVM, as it is no longer
> being actively developed from what I can see. The couple of crashes I
> have reported have gotten fixed pretty quickly but that is only because
> they were relatively simple. For the more complicated issues, I doubt we
> would have an easy time getting those fixed without more support. Until
> that happens (if it ever does), it probably isn't worth testing MIPS in
> any sort of continuous capacity aside from the few configurations we
> have working already (I test malta_defconfig + the generic 32r*
> configs).

My feeling is more that if nobody can be bothered to implement a
simple command line flag like this one, there is not much hope of them
caring about other bugs either.

         Arnd