[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Thomas Petazzoni]

Hello Alexey,

In commit 2b93fe5374ae4f8337e1f7ae32be4d72dd94cb3c, you added the
support for pre-built ARC toolchains from Synopsys, which was a very
useful change.

However, right now, I wonder if we should not get rid of those
toolchains: they use a uClibc configuration that is too different from
the default one of Buildroot, causing numerous build failures in the
autobuilders. Those failures are due to:

 * Missing libutil stub (UCLIBC_HAS_LIBUTIL option)

   http://autobuild.buildroot.org/results/ce3/ce39eb9b9ece0968563641fb2207099d1a37b191/

 * Missing program_invocation_name (UCLIBC_HAS_PROGRAM_INVOCATION_NAME
   option)

   http://autobuild.buildroot.org/results/154/1546d909e606daefd41b87dece94d642c0fdeba4/
   http://autobuild.buildroot.org/results/bd5/bd54581d7b0cc73bc501072d27e870a443dfce79/

 * Missing ifaddrd support (UCLIBC_SUPPORT_AI_ADDRCONFIG option)

   http://autobuild.buildroot.org/results/134/134e78ef1fa87f7fbf26c23ec5dfc68785d79613/

There are too many of those issues affecting too many packages for us
to reasonably patch all of them.

Could you work internally to prepare a new version of those toolchains,
that has at least these three additional options enabled?

In the mean time, unless you have an objection, I will mark those
toolchains as disabled in Buildroot.

Thanks in advance for your feedback,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Alexey Brodkin]

Hi Thomas,

On Fri, 2015-04-10 at 11:21 +0200, Thomas Petazzoni wrote:
> Hello Alexey,
> 
> In commit 2b93fe5374ae4f8337e1f7ae32be4d72dd94cb3c, you added the
> support for pre-built ARC toolchains from Synopsys, which was a very
> useful change.
> 
> However, right now, I wonder if we should not get rid of those
> toolchains: they use a uClibc configuration that is too different from
> the default one of Buildroot, causing numerous build failures in the
> autobuilders. Those failures are due to:
> 
>  * Missing libutil stub (UCLIBC_HAS_LIBUTIL option)
> 
>    http://autobuild.buildroot.org/results/ce3/ce39eb9b9ece0968563641fb2207099d1a37b191/
> 
>  * Missing program_invocation_name (UCLIBC_HAS_PROGRAM_INVOCATION_NAME
>    option)
> 
>    http://autobuild.buildroot.org/results/154/1546d909e606daefd41b87dece94d642c0fdeba4/
>    http://autobuild.buildroot.org/results/bd5/bd54581d7b0cc73bc501072d27e870a443dfce79/
> 
>  * Missing ifaddrd support (UCLIBC_SUPPORT_AI_ADDRCONFIG option)
> 
>    http://autobuild.buildroot.org/results/134/134e78ef1fa87f7fbf26c23ec5dfc68785d79613/
> 
> There are too many of those issues affecting too many packages for us
> to reasonably patch all of them.
> 
> Could you work internally to prepare a new version of those toolchains,
> that has at least these three additional options enabled?
> 
> In the mean time, unless you have an objection, I will mark those
> toolchains as disabled in Buildroot.

Looks like our pre-built tools didn't work as good as I expected. Indeed
there're so many options in uClibc that it's hard to match requirements
of all the different apps we build.

And at that point I'm not sure if there's a solution which satisfies
both sides (I mean Buildroot and Synopsys people). And that's not about
either side if bad or good.

Personally I'd like to have proven by many autobuild tests Buildroot
config as a defconfig for ARC. But there're following issues I may see:

 [1] Buildroot defconfigs for uClibc are not generated by "make
savedefconfig" which makes it very hard to understand which options are
enabled and which are disabled compared to defaults.

 [2] Buildroot defconfig may contain options (disabled or enabled
compared to defaults) that might be harmful (in run-time issues like
performance degradation etc). Remember missing
UCLIBC_HAS_STRING_GENERIC_OPT. This is I think a reminder of early days
of Buildroot. And so blind use of Buildroot defconfig as defconfig for
ARC might not work well enough.

So probably for now removal or hiding of AC pre-built tools is the
ultimate solution in short-term. The next release of our tools is
scheduled on June-July this year and I hope to put required changes in
defconfigs by that time.

Still I think we all may benefit from observed situation.

For starters why don't we run at least "uClibc-snapshot.config" through
"make savedefconfig". So we'll have only differences from defaults.

That's what I got running "make savedefconfig" on the current
"uClibc-snapshot.config" note it's only 40 lines in length compared to
256 lines in "uClibc-snapshot.config":
--->8---
DO_C99_MATH=y
KERNEL_HEADERS="/usr/src/linux/include"
# LDSO_CACHE_SUPPORT is not set
# UCLIBC_STATIC_LDCONFIG is not set
LDSO_RUNPATH=y
LINUXTHREADS_OLD=y
MALLOC_GLIBC_COMPAT=y
UCLIBC_HAS_OBSTACK=y
UCLIBC_HAS_UTMPX=y
UCLIBC_HAS_UTMP=y
UCLIBC_SUSV2_LEGACY=y
UCLIBC_SUSV3_LEGACY=y
UCLIBC_SUSV4_LEGACY=y
UCLIBC_HAS_PROGRAM_INVOCATION_NAME=y
UCLIBC_HAS_GETPT=y
UCLIBC_HAS_LIBUTIL=y
UCLIBC_HAS_OBSOLETE_BSD_SIGNAL=y
UCLIBC_USE_NETLINK=y
UCLIBC_SUPPORT_AI_ADDRCONFIG=y
UCLIBC_HAS_RESOLVER_SUPPORT=y
UCLIBC_HAS_LIBRESOLV_STUB=y
UCLIBC_HAS_LIBNSL_STUB=y
UCLIBC_HAS_CTYPE_CHECKED=y
UCLIBC_HAS_HEXADECIMAL_FLOATS=y
UCLIBC_HAS_GLIBC_CUSTOM_PRINTF=y
# UCLIBC_HAS_STDIO_GETC_MACRO is not set
# UCLIBC_HAS_STDIO_PUTC_MACRO is not set
UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE=y
UCLIBC_HAS_GLIBC_CUSTOM_STREAMS=y
UCLIBC_HAS_PRINTF_M_SPEC=y
# UCLIBC_HAS_REGEX_OLD is not set
# UCLIBC_HAS_FNMATCH_OLD is not set
UCLIBC_HAS_NFTW=y
UCLIBC_HAS_FTW=y
UCLIBC_HAS_GNU_GLOB=y
RUNTIME_PREFIX="/"
DEVEL_PREFIX="/usr/"
UCLIBC_HAS_SSP=y
UCLIBC_BUILD_NOW=y
# DOSTRIP is not set
--->8---

BTW that's funny what says header of the current
"uClibc-snapshot.config" even though it's year 2015 and snapshots are
far behind 0.9.33 :)
--->8---
# Automatically generated make config: don't edit
# Version: 0.9.32-git
# Fri Jul  9 22:31:59 2010
--->8---

-Alexey

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Arnout Vandecappelle]

On 17/04/15 10:49, Alexey Brodkin wrote:
>  [1] Buildroot defconfigs for uClibc are not generated by "make
> savedefconfig" which makes it very hard to understand which options are
> enabled and which are disabled compared to defaults.

 I think the only reason why this is the case is that uClibc didn't use to
support savedefconfig. AFAICS 0.9.33.2 still doesn't.


>  [2] Buildroot defconfig may contain options (disabled or enabled
> compared to defaults) that might be harmful (in run-time issues like
> performance degradation etc). Remember missing
> UCLIBC_HAS_STRING_GENERIC_OPT. This is I think a reminder of early days
> of Buildroot. And so blind use of Buildroot defconfig as defconfig for
> ARC might not work well enough.

 We're certainly open to improving the BR defconfigs!


 Regards,
 Arnout



-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Alexey Brodkin]

Hi Thomas,

On Fri, 2015-04-17 at 21:47 +0200, Arnout Vandecappelle wrote:
> On 17/04/15 10:49, Alexey Brodkin wrote:
> >  [1] Buildroot defconfigs for uClibc are not generated by "make
> > savedefconfig" which makes it very hard to understand which options are
> > enabled and which are disabled compared to defaults.
> 
>  I think the only reason why this is the case is that uClibc didn't use to
> support savedefconfig. AFAICS 0.9.33.2 still doesn't.
> 
> 
> >  [2] Buildroot defconfig may contain options (disabled or enabled
> > compared to defaults) that might be harmful (in run-time issues like
> > performance degradation etc). Remember missing
> > UCLIBC_HAS_STRING_GENERIC_OPT. This is I think a reminder of early days
> > of Buildroot. And so blind use of Buildroot defconfig as defconfig for
> > ARC might not work well enough.
> 
>  We're certainly open to improving the BR defconfigs!

Please note that we added missing options:
 * UCLIBC_HAS_UTMP=y
 * UCLIBC_HAS_UTMPX=y
 * UCLIBC_HAS_LIBUTIL=y
 * UCLIBC_USE_NETLINK=y
 * UCLIBC_SUPPORT_AI_ADDRCONFIG=y
 * UCLIBC_HAS_LIBNSL_STUB=y

in uClibc defconfigs for ARC so the next release of ARC toolchain will have them -
https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/commit/ab87ede75f9b9b837d579c8eeef3ab4f22fd45d1

As for remaining differences between Buildroot and ARC uClibc defconfigs
these are ones I may see (comparing
https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/blob/arc-mainline-dev/extra/Configs/defconfigs/arc/defconfig with "uClibc-snapshot.config" from http://patchwork.ozlabs.org/patch/462812/):

 [1] MALLOC_GLIBC_COMPAT=y
 [2] UCLIBC_HAS_GETPT=y
 [3] UCLIBC_HAS_CTYPE_CHECKED=y
 [4] UCLIBC_HAS_HEXADECIMAL_FLOATS=y
 [5] UCLIBC_HAS_GLIBC_CUSTOM_PRINTF=y
 [6] UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE=y
 [7] UCLIBC_HAS_PRINTF_M_SPEC=y
 [8] UCLIBC_HAS_SSP=y
 [9] UCLIBC_BUILD_NOW=y

Most of these options enable more compatibility with glibc or even
expand support for non-standard features like
UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE.

Do you think some of them are really important and we may want to add
them in ARC defconfigs?

-Alexey

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Thomas Petazzoni]

Alexey,

On Wed, 22 Apr 2015 10:47:03 +0000, Alexey Brodkin wrote:

> Please note that we added missing options:
>  * UCLIBC_HAS_UTMP=y
>  * UCLIBC_HAS_UTMPX=y
>  * UCLIBC_HAS_LIBUTIL=y
>  * UCLIBC_USE_NETLINK=y
>  * UCLIBC_SUPPORT_AI_ADDRCONFIG=y
>  * UCLIBC_HAS_LIBNSL_STUB=y

Great.

> in uClibc defconfigs for ARC so the next release of ARC toolchain will have them -
> https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/commit/ab87ede75f9b9b837d579c8eeef3ab4f22fd45d1
> 
> As for remaining differences between Buildroot and ARC uClibc defconfigs
> these are ones I may see (comparing
> https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/blob/arc-mainline-dev/extra/Configs/defconfigs/arc/defconfig with "uClibc-snapshot.config" from http://patchwork.ozlabs.org/patch/462812/):
> 
>  [1] MALLOC_GLIBC_COMPAT=y
>  [2] UCLIBC_HAS_GETPT=y
>  [3] UCLIBC_HAS_CTYPE_CHECKED=y
>  [4] UCLIBC_HAS_HEXADECIMAL_FLOATS=y
>  [5] UCLIBC_HAS_GLIBC_CUSTOM_PRINTF=y
>  [6] UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE=y
>  [7] UCLIBC_HAS_PRINTF_M_SPEC=y
>  [8] UCLIBC_HAS_SSP=y
>  [9] UCLIBC_BUILD_NOW=y
> 
> Most of these options enable more compatibility with glibc or even
> expand support for non-standard features like
> UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE.
> 
> Do you think some of them are really important and we may want to add
> them in ARC defconfigs?

I think the closer you are to the Buildroot uClibc defconfig, the less
compatibility issues we'll have. So unless you have a good reason to
not use the Buildroot uClibc defconfig, I would suggest to just use it.
Of course, feel free to *add* more features than this defconfig, but
try to not have less features.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Arnout Vandecappelle]

On 04/22/15 12:47, Alexey Brodkin wrote:
[snip]
> As for remaining differences between Buildroot and ARC uClibc defconfigs
> these are ones I may see (comparing
> https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/blob/arc-mainline-dev/extra/Configs/defconfigs/arc/defconfig with "uClibc-snapshot.config" from http://patchwork.ozlabs.org/patch/462812/):
> 
>  [1] MALLOC_GLIBC_COMPAT=y
>  [2] UCLIBC_HAS_GETPT=y
>  [3] UCLIBC_HAS_CTYPE_CHECKED=y
>  [4] UCLIBC_HAS_HEXADECIMAL_FLOATS=y
>  [5] UCLIBC_HAS_GLIBC_CUSTOM_PRINTF=y
>  [6] UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE=y
>  [7] UCLIBC_HAS_PRINTF_M_SPEC=y
>  [8] UCLIBC_HAS_SSP=y
>  [9] UCLIBC_BUILD_NOW=y
> 
> Most of these options enable more compatibility with glibc or even
> expand support for non-standard features like
> UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE.
> 
> Do you think some of them are really important and we may want to add
> them in ARC defconfigs?

 Well, basically the uClibc configs we have are just a configuration that was
created many many years ago. Occasionally some option is added when there are
important packages that need it, but it's not as if really a lot of thought goes
into these configs nowadays.

 Since you're doing active development with uClibc, I expect that there are some
options that you encounter that actually improve the quality of the resulting
library, it would be good to get patches that enable them (of course with
sufficient explanation _why_ it's a good idea to enable it).

 Your commit 826f315d is a great example of this.


 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Disabling Synopsys ARC pre-built toolchains in Buildroot

[Waldemar Brodkorb]

Hi Alexey,
Alexey Brodkin wrote,

> Hi Thomas,
> 
> On Fri, 2015-04-17 at 21:47 +0200, Arnout Vandecappelle wrote:
> > On 17/04/15 10:49, Alexey Brodkin wrote:
> > >  [1] Buildroot defconfigs for uClibc are not generated by "make
> > > savedefconfig" which makes it very hard to understand which options are
> > > enabled and which are disabled compared to defaults.
> > 
> >  I think the only reason why this is the case is that uClibc didn't use to
> > support savedefconfig. AFAICS 0.9.33.2 still doesn't.
> > 
> > 
> > >  [2] Buildroot defconfig may contain options (disabled or enabled
> > > compared to defaults) that might be harmful (in run-time issues like
> > > performance degradation etc). Remember missing
> > > UCLIBC_HAS_STRING_GENERIC_OPT. This is I think a reminder of early days
> > > of Buildroot. And so blind use of Buildroot defconfig as defconfig for
> > > ARC might not work well enough.
> > 
> >  We're certainly open to improving the BR defconfigs!
> 
> Please note that we added missing options:
>  * UCLIBC_HAS_UTMP=y
>  * UCLIBC_HAS_UTMPX=y
>  * UCLIBC_HAS_LIBUTIL=y
>  * UCLIBC_USE_NETLINK=y
>  * UCLIBC_SUPPORT_AI_ADDRCONFIG=y
>  * UCLIBC_HAS_LIBNSL_STUB=y
> 
> in uClibc defconfigs for ARC so the next release of ARC toolchain will have them -
> https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/commit/ab87ede75f9b9b837d579c8eeef3ab4f22fd45d1
> 
> As for remaining differences between Buildroot and ARC uClibc defconfigs
> these are ones I may see (comparing
> https://github.com/foss-for-synopsys-dwc-arc-processors/uClibc/blob/arc-mainline-dev/extra/Configs/defconfigs/arc/defconfig with "uClibc-snapshot.config" from http://patchwork.ozlabs.org/patch/462812/):
> 
>  [1] MALLOC_GLIBC_COMPAT=y
>  [2] UCLIBC_HAS_GETPT=y
>  [3] UCLIBC_HAS_CTYPE_CHECKED=y
>  [4] UCLIBC_HAS_HEXADECIMAL_FLOATS=y
>  [5] UCLIBC_HAS_GLIBC_CUSTOM_PRINTF=y
>  [6] UCLIBC_HAS_FOPEN_EXCLUSIVE_MODE=y
>  [7] UCLIBC_HAS_PRINTF_M_SPEC=y
>  [8] UCLIBC_HAS_SSP=y
>  [9] UCLIBC_BUILD_NOW=y
> 
> Do you think some of them are really important and we may want to add
> them in ARC defconfigs?

I think MALLOC_GLIBC_COMPAT should be enabled. GNU libc and musl
behaves in the same way, only uClibc does it another way.
uClibc-ng-master already removed this option, because it will be the
default for the next major release.

best regards
 Waldemar