More C errors by default in GCC 14 (implicit function declarations etc.)

More C errors by default in GCC 14 (implicit function declarations etc.)

[Florian Weimer]

TL;DR: I want to propose a GCC 14 change which will impact
distributions, so I'd like to gather some feedback from OpenSUSE.

Clang has disabled support for a few historic C features by default over
the last few releases.  This mirrors a process that Apple has begun in
Xcode even earlier (perhaps motivated in part by their AArch64 Darwin
ABI, which is pretty much incompatible with some of the C89-only
features).

These changes bring real benefits to C programmers because errors are
much harder to miss during the build than warnings.  In many cases, the
compiler is not able to generate correct code when such issues are
present, and programmers who look at the generated machine code suspect
a compiler bug.  And all this happens because they missed a warning.  So
we want this change for GCC, too.

On the other hand, many distributions use GCC as the system compiler,
and there the focus is not so much on developing software, but building
the sources as they exist today.  It's somewhat different the usual GCC
C++ updates (both language changes and libstdc++ header changes) because
it impacts pre-build feature probing (mostly autoconf).  If that happens
and the probe goes wrong due to a new compiler error, it's possible that
a build still succeeds, passes its test suite, but lacks the intended
feature or ABI because parts got automatically disabled due to the
failing configure check.  With C++ transitions, that seems more rare
(C++ programs—if they use autoconf—often run the checks with the C
compiler).

Specifically, I'm investigating the following changes:

* -Werror=implicit-function-declaration: Functions can no longer be
   called without be declaring first.  Fixing this may need additional
   prototypes in package header files, or inclusion of additional header
   files (both package-specific and system headers).

* -Werror=implict-int: int types can no longer be omitted in old-style
   function definitions, function return types, or variable declarations
   or definitions.  Fixing that involves adding the int type (or the
   correct type if it is not actually int).  If there is already a
   matching declaration in scope that has a different type, that needs
   to be resolved somehow, too.

* (tentative) -Werror=int-conversion: Conversion between pointer and
  integer types without an explicit cast is now a compiler error.
  Usually fixed by one of the two things above.  I do not have data yet
  how many other cases remain after the initial issues are fixed, but
  should have that in the coming weeks.  (Quite frankly, I'm amazed that
  we created 64-bit ports without making this an error.)

* (very tentative) -Werror=incompatible-pointer-types: GCC will no
  longer automatically convert between pointer values of unrelated
  pointer types (except when one of them is void * or its qualified
  versions).  The fallout from this could be quite large, I do not have
  data yet.  Clang does this for function pointer types only (probably
  based on their ABI issues), but I'm not sure if it's a reasonable
  distinction for our ABIs (the ppc64le cases I've seen had explicit
  casts and no warnings).

* For -Wdiscarded-qualifies (e.g., using const pointers as non-const),
  and -Wpointe-rsign (using char * as unsigned char *) I do not have any
  plans.

I want to propose at least the first two for GCC 14.

The exact mechanism how the default is changed may not be -Werror=,
perhaps something along the lines of -fpermissive for C++.  The C89
modes (-std=gnu89 etc.) will likely still enable all these features
(even if they are not part of C89).  As an opt-out mechanism, -std=gnu89
is insufficient because there are packages out there which use features
which are C99-and-later-only in GCC (predominantly C99-style inlining, I
believe) together with implicit-int/implicit-function-declaration.

For Fedora, we are using an instrumented compiler to find packages that
need fixing.  More details on the process are here (but please ask if
you are interested in specifics):

  <https://fedoraproject.org/wiki/Changes/PortingToModernC>
  <https://fedoraproject.org/wiki/Toolchain/PortingToModernC>

The numbers so far don't look great, but are manageable.  Fedora has
23,101 source package last time a looked.  We have fixed 796 of them,
and 85 are still pending investigation (with 5-10 false positives
expected remaining).  This puts the per-package failure rate at 3.8%.  I
don't have data on silent failures; most issues do not seem to be
silent, and fully-silent packages are rare.  The silent output-changing
issues definitely exist, they are usually accompanied by something else.
Those 3.8% also include some packages which we fixed by removing C89
constructs, but where the relevant autoconf tests failed for other
reasons.

Fedora would be hit pretty hard if we made the GCC switch without this
preparation because we do a mass rebuild of the entire distribution
right after importing a new GCC upstream release.  I have considered
automating some of the autoconf updates, but usually it's some generic
autoconf issue (long since fixed in autoconf) plus a package-specific
issue, so that doesn't seem to be particularly helpful.

The changes we have made in Fedora are captured here:

  <https://gitlab.com/fweimer-rh/fedora-modernc/-/tree/main/pkg>

In general, if there is an upstream reference for change (bug tracker,
mailing list), we have not filed downstream bugs.  Neither if it's
something that is the result of an old autoconf bug.  I don't know how
useful this data is going to be for other distributions.

Gentoo has been fixing various packages for building with Clang, which
covers a superset of the issues that need to be addressed:

  [TRACKER] Support LLVM/Clang as alternative system compiler 
  <https://bugs.gentoo.org/showdependencytree.cgi?id=408963&hide_resolved=0>

IIRC, Gentoo has its own mechanism to detect silent build breakage, but
I think it's mostly focused on autoconf, so it's less comprehensive, and
also fixes the stuff that is actually relevant to the distribution.

Like the Fedora effort, they try to upstream patches (if an upstream is
still around).  Xcode/Homebrew/Macports users have upstreamed some
patches as well, but perhaps less consistently so.  Most upstreams are
receptive to the changes.  If they reject them, it's mostly becaue of
CLA processes.  But for Fedora, there's a large overlap between impacted
packages and packages without an active upstream maintainer, which is
perhaps not unexpected.


I would appreciate some discussion on the OpenSUSE impact.  I assume
OpenSUSE does mass rebuilds after GCC rebases, a bit like Fedora?  How
much time do you have until GCC 14 lands in at least some repositories?
In Fedora, we tend to apply the fixes even before upstream acceptance,
and do not wait until they land through routine rebases (which happen
only once individual package maintainers decide to do them).

Do you think OpenSUSE could cope with a transition in GCC 14?

Thanks,
Florian

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Neal Gompa]

On Tue, Apr 18, 2023 at 8:36 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> TL;DR: I want to propose a GCC 14 change which will impact
> distributions, so I'd like to gather some feedback from OpenSUSE.
>
> Clang has disabled support for a few historic C features by default over
> the last few releases.  This mirrors a process that Apple has begun in
> Xcode even earlier (perhaps motivated in part by their AArch64 Darwin
> ABI, which is pretty much incompatible with some of the C89-only
> features).
>
> These changes bring real benefits to C programmers because errors are
> much harder to miss during the build than warnings.  In many cases, the
> compiler is not able to generate correct code when such issues are
> present, and programmers who look at the generated machine code suspect
> a compiler bug.  And all this happens because they missed a warning.  So
> we want this change for GCC, too.
>
> On the other hand, many distributions use GCC as the system compiler,
> and there the focus is not so much on developing software, but building
> the sources as they exist today.  It's somewhat different the usual GCC
> C++ updates (both language changes and libstdc++ header changes) because
> it impacts pre-build feature probing (mostly autoconf).  If that happens
> and the probe goes wrong due to a new compiler error, it's possible that
> a build still succeeds, passes its test suite, but lacks the intended
> feature or ABI because parts got automatically disabled due to the
> failing configure check.  With C++ transitions, that seems more rare
> (C++ programs—if they use autoconf—often run the checks with the C
> compiler).
>
> Specifically, I'm investigating the following changes:
>
> * -Werror=implicit-function-declaration: Functions can no longer be
>    called without be declaring first.  Fixing this may need additional
>    prototypes in package header files, or inclusion of additional header
>    files (both package-specific and system headers).
>
> * -Werror=implict-int: int types can no longer be omitted in old-style
>    function definitions, function return types, or variable declarations
>    or definitions.  Fixing that involves adding the int type (or the
>    correct type if it is not actually int).  If there is already a
>    matching declaration in scope that has a different type, that needs
>    to be resolved somehow, too.
>
> * (tentative) -Werror=int-conversion: Conversion between pointer and
>   integer types without an explicit cast is now a compiler error.
>   Usually fixed by one of the two things above.  I do not have data yet
>   how many other cases remain after the initial issues are fixed, but
>   should have that in the coming weeks.  (Quite frankly, I'm amazed that
>   we created 64-bit ports without making this an error.)
>
> * (very tentative) -Werror=incompatible-pointer-types: GCC will no
>   longer automatically convert between pointer values of unrelated
>   pointer types (except when one of them is void * or its qualified
>   versions).  The fallout from this could be quite large, I do not have
>   data yet.  Clang does this for function pointer types only (probably
>   based on their ABI issues), but I'm not sure if it's a reasonable
>   distinction for our ABIs (the ppc64le cases I've seen had explicit
>   casts and no warnings).
>
> * For -Wdiscarded-qualifies (e.g., using const pointers as non-const),
>   and -Wpointe-rsign (using char * as unsigned char *) I do not have any
>   plans.
>
> I want to propose at least the first two for GCC 14.
>
> The exact mechanism how the default is changed may not be -Werror=,
> perhaps something along the lines of -fpermissive for C++.  The C89
> modes (-std=gnu89 etc.) will likely still enable all these features
> (even if they are not part of C89).  As an opt-out mechanism, -std=gnu89
> is insufficient because there are packages out there which use features
> which are C99-and-later-only in GCC (predominantly C99-style inlining, I
> believe) together with implicit-int/implicit-function-declaration.
>
> For Fedora, we are using an instrumented compiler to find packages that
> need fixing.  More details on the process are here (but please ask if
> you are interested in specifics):
>
>   <https://fedoraproject.org/wiki/Changes/PortingToModernC>
>   <https://fedoraproject.org/wiki/Toolchain/PortingToModernC>
>
> The numbers so far don't look great, but are manageable.  Fedora has
> 23,101 source package last time a looked.  We have fixed 796 of them,
> and 85 are still pending investigation (with 5-10 false positives
> expected remaining).  This puts the per-package failure rate at 3.8%.  I
> don't have data on silent failures; most issues do not seem to be
> silent, and fully-silent packages are rare.  The silent output-changing
> issues definitely exist, they are usually accompanied by something else.
> Those 3.8% also include some packages which we fixed by removing C89
> constructs, but where the relevant autoconf tests failed for other
> reasons.
>
> Fedora would be hit pretty hard if we made the GCC switch without this
> preparation because we do a mass rebuild of the entire distribution
> right after importing a new GCC upstream release.  I have considered
> automating some of the autoconf updates, but usually it's some generic
> autoconf issue (long since fixed in autoconf) plus a package-specific
> issue, so that doesn't seem to be particularly helpful.
>
> The changes we have made in Fedora are captured here:
>
>   <https://gitlab.com/fweimer-rh/fedora-modernc/-/tree/main/pkg>
>
> In general, if there is an upstream reference for change (bug tracker,
> mailing list), we have not filed downstream bugs.  Neither if it's
> something that is the result of an old autoconf bug.  I don't know how
> useful this data is going to be for other distributions.
>
> Gentoo has been fixing various packages for building with Clang, which
> covers a superset of the issues that need to be addressed:
>
>   [TRACKER] Support LLVM/Clang as alternative system compiler
>   <https://bugs.gentoo.org/showdependencytree.cgi?id=408963&hide_resolved=0>
>
> IIRC, Gentoo has its own mechanism to detect silent build breakage, but
> I think it's mostly focused on autoconf, so it's less comprehensive, and
> also fixes the stuff that is actually relevant to the distribution.
>
> Like the Fedora effort, they try to upstream patches (if an upstream is
> still around).  Xcode/Homebrew/Macports users have upstreamed some
> patches as well, but perhaps less consistently so.  Most upstreams are
> receptive to the changes.  If they reject them, it's mostly becaue of
> CLA processes.  But for Fedora, there's a large overlap between impacted
> packages and packages without an active upstream maintainer, which is
> perhaps not unexpected.
>
>
> I would appreciate some discussion on the OpenSUSE impact.  I assume
> OpenSUSE does mass rebuilds after GCC rebases, a bit like Fedora?  How
> much time do you have until GCC 14 lands in at least some repositories?
> In Fedora, we tend to apply the fixes even before upstream acceptance,
> and do not wait until they land through routine rebases (which happen
> only once individual package maintainers decide to do them).
>
> Do you think OpenSUSE could cope with a transition in GCC 14?
>

openSUSE does rebuilds of the whole distribution on a fairly regular
basis, as reverse dependencies of updated packages get rebuilt
automatically. When glibc or GCC get updated, the whole reverse
dependency chain gets rebuilt, which is effectively the whole
distribution modulo data-only RPMs.

I know that we trigger mass rebuilds directly sometimes, but it is not
a thing we do. We don't have to, since the build system takes care of
it for us most of the time.

Things fall out of openSUSE fairly aggressively because of this, so I
suspect we'll run into fewer problems than Fedora did.

(As an aside, automated rebuilds of reverse dependencies makes life
tremendously easier, you should try it!)


-- 
真実はいつも一つ！/ Always, there's only one truth!

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Jan Engelhardt]

On Tuesday 2023-04-18 15:19, Neal Gompa wrote:
>On Tue, Apr 18, 2023 at 8:36 AM Florian Weimer <fweimer@redhat.com> wrote:
>>
>> TL;DR: I want to propose a GCC 14 change which will impact
>> distributions, so I'd like to gather some feedback from OpenSUSE.
>>
>> Specifically, I'm investigating the following changes:
>>
>> * -Werror=implicit-function-declaration
>> * -Werror=implict-int
>> * (tentative) -Werror=int-conversion
>> * (very tentative) -Werror=incompatible-pointer-types
>> * For -Wdiscarded-qualifies
>>
>> I want to propose at least the first two for GCC 14.

Might as well enable all of it at once, then the community only needs
one pass per defective software.

>> The numbers so far don't look great, but are manageable.  Fedora has
>> 23,101 source package last time a looked.  We have fixed 796 of them,

3.8% is actually really great for syntax-related changes,
reproducible-builds (in openSUSE) also has a nay ratio of ~3.4%,
but of course r-b is more difficult to fix I think.

>> I would appreciate some discussion on the OpenSUSE impact.  I assume
>> OpenSUSE does mass rebuilds after GCC rebases, a bit like Fedora?
>
>openSUSE does rebuilds of the whole distribution on a fairly regular
>basis

There is a also staging project (ironically named
openSUSE:Factory:Staging:Gcc7) where a new gcc (not 7) gets to
test-build a sizable portion of the distro before promoting it in
openSUSE:Factory and before causing the super rebuild.

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Neal Gompa]

On Tue, Apr 18, 2023 at 9:42 AM Jan Engelhardt <jengelh@inai.de> wrote:
>
>
> On Tuesday 2023-04-18 15:19, Neal Gompa wrote:
> >On Tue, Apr 18, 2023 at 8:36 AM Florian Weimer <fweimer@redhat.com> wrote:
> >>
> >> TL;DR: I want to propose a GCC 14 change which will impact
> >> distributions, so I'd like to gather some feedback from OpenSUSE.
> >>
> >> Specifically, I'm investigating the following changes:
> >>
> >> * -Werror=implicit-function-declaration
> >> * -Werror=implict-int
> >> * (tentative) -Werror=int-conversion
> >> * (very tentative) -Werror=incompatible-pointer-types
> >> * For -Wdiscarded-qualifies
> >>
> >> I want to propose at least the first two for GCC 14.
>
> Might as well enable all of it at once, then the community only needs
> one pass per defective software.
>

I would also like to reiterate this point. The worst thing you can do
is slow-roll changes like this. Being punched in the face once is
better than being punched in the face several times in succession.
Having to deal with this in Fedora, I would really rather not do it
over and over and over if we can do it all at once.

(Yes, I really feel like dealing with these changes is equivalent to
getting punched in the face. I dealt with them far too much with my
C++ packages in Fedora.)

> >> The numbers so far don't look great, but are manageable.  Fedora has
> >> 23,101 source package last time a looked.  We have fixed 796 of them,
>
> 3.8% is actually really great for syntax-related changes,
> reproducible-builds (in openSUSE) also has a nay ratio of ~3.4%,
> but of course r-b is more difficult to fix I think.
>

Build reproducibility (as defined by Debian, which is what openSUSE
follows) also has disputed usefulness among the broader community,
especially upstreams. That makes it harder for things to be fixed in
the first place.

> >> I would appreciate some discussion on the OpenSUSE impact.  I assume
> >> OpenSUSE does mass rebuilds after GCC rebases, a bit like Fedora?
> >
> >openSUSE does rebuilds of the whole distribution on a fairly regular
> >basis
>
> There is a also staging project (ironically named
> openSUSE:Factory:Staging:Gcc7) where a new gcc (not 7) gets to
> test-build a sizable portion of the distro before promoting it in
> openSUSE:Factory and before causing the super rebuild.

Super rebuilds are fun though. :)



--
真実はいつも一つ！/ Always, there's only one truth!

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 2469 bytes --]


Neal Gompa <ngompa13@gmail.com> writes:

> On Tue, Apr 18, 2023 at 9:42 AM Jan Engelhardt <jengelh@inai.de> wrote:
>>
>>
>> On Tuesday 2023-04-18 15:19, Neal Gompa wrote:
>> >On Tue, Apr 18, 2023 at 8:36 AM Florian Weimer <fweimer@redhat.com> wrote:
>> >>
>> >> TL;DR: I want to propose a GCC 14 change which will impact
>> >> distributions, so I'd like to gather some feedback from OpenSUSE.
>> >>
>> >> Specifically, I'm investigating the following changes:
>> >>
>> >> * -Werror=implicit-function-declaration
>> >> * -Werror=implict-int
>> >> * (tentative) -Werror=int-conversion
>> >> * (very tentative) -Werror=incompatible-pointer-types
>> >> * For -Wdiscarded-qualifies
>> >>
>> >> I want to propose at least the first two for GCC 14.
>>
>> Might as well enable all of it at once, then the community only needs
>> one pass per defective software.
>>
>
> I would also like to reiterate this point. The worst thing you can do
> is slow-roll changes like this. Being punched in the face once is
> better than being punched in the face several times in succession.
> Having to deal with this in Fedora, I would really rather not do it
> over and over and over if we can do it all at once.
>
> (Yes, I really feel like dealing with these changes is equivalent to
> getting punched in the face. I dealt with them far too much with my
> C++ packages in Fedora.)

Yeah, I can understand that point, and I'd like to do more, *but* that's
likely to only be feasible if we have at least one person from openSUSE
(ideally more!) championing it and doing tests with some of the flags
(Florian and I can share our setups - they're quite different to each
other, etc if needed).

It (broadly) consists of doing a full rebuild with some strict flags
and deciding on a method of obtaining the errors and capturing them
(might use a compiler wrapper, might use a patched GCC like Fedora is,
might try to grep logs in the build dir afterwards, etc).

If we had someone else helping out with this with some of these
additional warnings, it becomes more realistic for us to try target it.

In Gentoo, we're already partly doing this because Clang 16 made most
of these changes already, but I'm prioritising the bugs with warnings
that I know GCC is likely to make fatal soon too.

From what I've seen, -Wint-conversion and -Wincompatible-pointer-types
are much rarer, but they also sometimes require more analysis to fix.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Bernhard Voelker]

On 4/18/23 15:19, Neal Gompa wrote:
>> Do you think OpenSUSE could cope with a transition in GCC 14?
>>
> openSUSE does rebuilds of the whole distribution on a fairly regular
> basis, as reverse dependencies of updated packages get rebuilt
> automatically.

One aspect is that upstream developers are using distros like Fedora and OpenSUSE,
so the first step is to provide GCC14 as alternative GCC version there.
With that, upstream developers have a chance to fix those issues ... upstream.

To put it the other way round:
I don't see that too many packages would get or should be fixed as downstream patches only.
Yet the downstream build environments are of great help as well.

Have a nice day,
Berny

Re: More C errors by default in GCC 14 (implicit function declarations etc.)

[Neal Gompa]

On Tue, Apr 18, 2023 at 10:41 AM Sam James <sam@gentoo.org> wrote:
>
>
> Neal Gompa <ngompa13@gmail.com> writes:
>
> > On Tue, Apr 18, 2023 at 9:42 AM Jan Engelhardt <jengelh@inai.de> wrote:
> >>
> >>
> >> On Tuesday 2023-04-18 15:19, Neal Gompa wrote:
> >> >On Tue, Apr 18, 2023 at 8:36 AM Florian Weimer <fweimer@redhat.com> wrote:
> >> >>
> >> >> TL;DR: I want to propose a GCC 14 change which will impact
> >> >> distributions, so I'd like to gather some feedback from OpenSUSE.
> >> >>
> >> >> Specifically, I'm investigating the following changes:
> >> >>
> >> >> * -Werror=implicit-function-declaration
> >> >> * -Werror=implict-int
> >> >> * (tentative) -Werror=int-conversion
> >> >> * (very tentative) -Werror=incompatible-pointer-types
> >> >> * For -Wdiscarded-qualifies
> >> >>
> >> >> I want to propose at least the first two for GCC 14.
> >>
> >> Might as well enable all of it at once, then the community only needs
> >> one pass per defective software.
> >>
> >
> > I would also like to reiterate this point. The worst thing you can do
> > is slow-roll changes like this. Being punched in the face once is
> > better than being punched in the face several times in succession.
> > Having to deal with this in Fedora, I would really rather not do it
> > over and over and over if we can do it all at once.
> >
> > (Yes, I really feel like dealing with these changes is equivalent to
> > getting punched in the face. I dealt with them far too much with my
> > C++ packages in Fedora.)
>
> Yeah, I can understand that point, and I'd like to do more, *but* that's
> likely to only be feasible if we have at least one person from openSUSE
> (ideally more!) championing it and doing tests with some of the flags
> (Florian and I can share our setups - they're quite different to each
> other, etc if needed).
>
> It (broadly) consists of doing a full rebuild with some strict flags
> and deciding on a method of obtaining the errors and capturing them
> (might use a compiler wrapper, might use a patched GCC like Fedora is,
> might try to grep logs in the build dir afterwards, etc).
>
> If we had someone else helping out with this with some of these
> additional warnings, it becomes more realistic for us to try target it.
>
> In Gentoo, we're already partly doing this because Clang 16 made most
> of these changes already, but I'm prioritising the bugs with warnings
> that I know GCC is likely to make fatal soon too.
>
> From what I've seen, -Wint-conversion and -Wincompatible-pointer-types
> are much rarer, but they also sometimes require more analysis to fix.

My point is that if you drag all these changes out, it's going to be
harder to make it successful. Let's just do them all at once and get
everyone fixing everything all at once. That scales better and takes
less overall effort.

If you feel like you're going to have a hard time here, compressing
the time range to uplift C code is going to go better.


-- 
真実はいつも一つ！/ Always, there's only one truth!