[PATCH 1/2] Remove noreturn function pointers in usage.c

[PATCH 1/2] Remove noreturn function pointers in usage.c

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Due to a bug in gcc 4.6+ it can crash when doing profile feedback
with a noreturn function pointer

(http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299)

Remove the NORETURNs from the die functions for now to work
around this. Doesn't seem to make any difference.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 usage.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/usage.c b/usage.c
index b5e67e3..4045574 100644
--- a/usage.c
+++ b/usage.c
@@ -12,13 +12,13 @@ void vreportf(const char *prefix, const char *err, va_list params)
 	fprintf(stderr, "%s%s\n", prefix, msg);
 }
 
-static NORETURN void usage_builtin(const char *err, va_list params)
+static  void usage_builtin(const char *err, va_list params)
 {
 	vreportf("usage: ", err, params);
 	exit(129);
 }
 
-static NORETURN void die_builtin(const char *err, va_list params)
+static  void die_builtin(const char *err, va_list params)
 {
 	vreportf("fatal: ", err, params);
 	exit(128);
@@ -36,8 +36,8 @@ static void warn_builtin(const char *warn, va_list params)
 
 /* If we are in a dlopen()ed .so write to a global variable would segfault
  * (ugh), so keep things static. */
-static NORETURN_PTR void (*usage_routine)(const char *err, va_list params) = usage_builtin;
-static NORETURN_PTR void (*die_routine)(const char *err, va_list params) = die_builtin;
+static void (*usage_routine)(const char *err, va_list params) = usage_builtin;
+static void (*die_routine)(const char *err, va_list params) = die_builtin;
 static void (*error_routine)(const char *err, va_list params) = error_builtin;
 static void (*warn_routine)(const char *err, va_list params) = warn_builtin;
 
-- 
1.7.4.4

[PATCH 2/2] Add profile feedback build to git

[Andi Kleen]

From: Andi Kleen <ak@linux.intel.com>

Add a gcc profile feedback build option "profile-all" to the
main Makefile. It simply runs the test suite to generate feedback
data and the recompiles the main executables with that. The basic
structure is similar to the existing gcov code.

gcc is often able to generate better code with profile feedback
data. The training load also doesn't need to be too similar
to the actual load, it still gives benefits.

The test suite run is unfortunately quite long. It would
be good to find a suitable subset that runs faster and still
gives reasonable feedback.

For now the test suite runs single threaded (I had some
trouble running the test suite with -jX)

I tested it with git gc and git blame kernel/sched.c on a Linux
kernel tree. For gc I get about 2.7% improvement in wall clock
time by using the feedback build, for blame about 2.4%.
That's not gigantic, but not shabby either for a very small patch.

If anyone has any favourite CPU intensive git benchmarks feel
free to try them too.

I hope distributors will switch to use a feedback build in their
packages.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
---
 Makefile |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index e40ac0c..85e2679 100644
--- a/Makefile
+++ b/Makefile
@@ -2486,3 +2486,20 @@ cover_db: coverage-report
 
 cover_db_html: cover_db
 	cover -report html -outputdir cover_db_html cover_db
+
+### profile feedback build
+#
+.PHONY: profile-all profile-clean
+
+PROFILE_GEN_CFLAGS := $(CFLAGS) -fprofile-generate
+PROFILE_USE_CFLAGS := $(CFLAGS) -fprofile-use -fprofile-correction
+
+profile-clean:
+	$(RM) $(addsuffix *.gcda,$(object_dirs))
+	$(RM) $(addsuffix *.gcno,$(object_dirs))
+
+profile-all: profile-clean
+	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" all
+	$(MAKE) CFLAGS="$(PROFILE_GEN_CFLAGS)" -j1 test
+	$(MAKE) CFLAGS="$(PROFILE_USE_CFLAGS)" all
+	
-- 
1.7.4.4

Re: [PATCH 1/2] Remove noreturn function pointers in usage.c

[Junio C Hamano]

Andi Kleen <andi@firstfloor.org> writes:

> From: Andi Kleen <ak@linux.intel.com>
>
> Due to a bug in gcc 4.6+ it can crash when doing profile feedback
> with a noreturn function pointer
>
> (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299)
>
> Remove the NORETURNs from the die functions for now to work
> around this. Doesn't seem to make any difference.
>
> Signed-off-by: Andi Kleen <ak@linux.intel.com>

I would expect a better patch from a well respected kernel person, though.

 - There are many more NORETURN and NORETURN_PTR in the code, and the
   proposed commit log message does not explain why these two are the only
   ones that are problematic and needs to be worked around. It does not
   guide other people who might want to add NORETURN/NORETURN_PTR when
   deciding if their change would break the "fix" this change brought in.

 - Potential impact to people who do not use Gcc 4.6 with profile feedback
   is not explained away well, except for "Doesn't seem to make any
   difference."

 - If other NORETURN/NORETURN_PTR could/should also go (I don't know due
   to the first bullet point above) when using the problematic compiler
   with the profile feedback feature, wouldn't it be a better workaround
   would be to introduce a Makefile variable to ask git-compat-util.h to
   make these two a no-op, perhaps?

A patch to do so may look like this.

I did not like the triple negation "make NO_NORETURN=NoThanks" and wanted
to name this AVOID_NORETURN instead, but decided to go with other existing
Makefile variables.


 Makefile          |    6 ++++++
 git-compat-util.h |    2 +-
 2 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/Makefile b/Makefile
index d2e2ea1..70c814c 100644
--- a/Makefile
+++ b/Makefile
@@ -144,6 +144,9 @@ all::
 # that tells runtime paths to dynamic libraries;
 # "-Wl,-rpath=/path/lib" is used instead.
 #
+# Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback,
+# as the compiler can crash (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299)
+#
 # Define USE_NSEC below if you want git to care about sub-second file mtimes
 # and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and
 # it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely
@@ -1349,6 +1352,9 @@ endif
 ifdef USE_ST_TIMESPEC
 	BASIC_CFLAGS += -DUSE_ST_TIMESPEC
 endif
+ifdef NO_NORETURN
+	BASIC_CFLAGS += -DNO_NORETURN
+endif
 ifdef NO_NSEC
 	BASIC_CFLAGS += -DNO_NSEC
 endif
diff --git a/git-compat-util.h b/git-compat-util.h
index 40498b3..13bc26f 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -218,7 +218,7 @@ extern char *gitbasename(char *);
 #if __HP_cc >= 61000
 #define NORETURN __attribute__((noreturn))
 #define NORETURN_PTR
-#elif defined(__GNUC__)
+#elif defined(__GNUC__) && !defined(NO_NORETURN)
 #define NORETURN __attribute__((__noreturn__))
 #define NORETURN_PTR __attribute__((__noreturn__))
 #elif defined(_MSC_VER)

Re: [PATCH 1/2] Remove noreturn function pointers in usage.c

[Andi Kleen]

>  - There are many more NORETURN and NORETURN_PTR in the code, and the
>    proposed commit log message does not explain why these two are the only
>    ones that are problematic and needs to be worked around. It does not
>    guide other people who might want to add NORETURN/NORETURN_PTR when
>    deciding if their change would break the "fix" this change brought in.

This was the only place where it crashed the compiler.

I don't have a good criterium to decide which cases do crash the compiler
or not except for trying it.

But I believe the crash is relatively unlikely (needs quite a lot of conditions
to line up), so it doesn't deserve extensive changes all over.

> 
>  - Potential impact to people who do not use Gcc 4.6 with profile feedback
>    is not explained away well, except for "Doesn't seem to make any
>    difference."

I merely went by "there are no new warnings" (I assume that's the main
motivation)

> 
>  - If other NORETURN/NORETURN_PTR could/should also go (I don't know due
>    to the first bullet point above) when using the problematic compiler
>    with the profile feedback feature, wouldn't it be a better workaround
>    would be to introduce a Makefile variable to ask git-compat-util.h to
>    make these two a no-op, perhaps?

I don't think we need to remove the others for now.

> 
> A patch to do so may look like this.
> 
> I did not like the triple negation "make NO_NORETURN=NoThanks" and wanted
> to name this AVOID_NORETURN instead, but decided to go with other existing
> Makefile variables.

Given the explanation above (I can update the description with that)
do you still want the complete disabling?

-Andi

Re: [PATCH 1/2] Remove noreturn function pointers in usage.c

[Jeff King]

On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote:

> >  - Potential impact to people who do not use Gcc 4.6 with profile feedback
> >    is not explained away well, except for "Doesn't seem to make any
> >    difference."
> 
> I merely went by "there are no new warnings" (I assume that's the main
> motivation)

On your compiler and settings, perhaps. With your patch I get:

  usage.c: In function ‘die’:
  usage.c:70:1: error: ‘noreturn’ function does return [-Werror]

And rightfully so:

void NORETURN die(const char *err, ...)
{
        va_list params;

        va_start(params, err);
        die_routine(err, params);
        va_end(params);
}

You've stripped the NORETURN from die_routine, so of course it looks like we
end up returning.

This is with:

  $ gcc --version | head -n 1
  gcc (Debian 4.6.0-11) 4.6.1 20110604 (prerelease)

-Peff

Re: [PATCH 1/2] Remove noreturn function pointers in usage.c

[Andi Kleen]

On Thu, Jun 09, 2011 at 01:52:24AM -0400, Jeff King wrote:
> On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote:
> 
> > >  - Potential impact to people who do not use Gcc 4.6 with profile feedback
> > >    is not explained away well, except for "Doesn't seem to make any
> > >    difference."
> > 
> > I merely went by "there are no new warnings" (I assume that's the main
> > motivation)
> 
> On your compiler and settings, perhaps. With your patch I get:
> 
>   usage.c: In function ‘die’:
>   usage.c:70:1: error: ‘noreturn’ function does return [-Werror]

Ok.  Hmm, all I can say it compiled here.

Ok then we have to remove it. I didn't really like Junio's approach
to only do it for a single file because that would break with LTO / link
time optimization which requires declarations to match between 
translation units.

Maybe it's better to stick an extra exit() at the end to shut up
the extra warning. I'll do that I guess.

BTW 4.6.2 or so will have the problem fixed.

-Andi

Re: [PATCH 1/2] Remove noreturn function pointers in usage.c

[Erik Faye-Lund]

On Thu, Jun 9, 2011 at 8:31 AM, Andi Kleen <andi@firstfloor.org> wrote:
> On Thu, Jun 09, 2011 at 01:52:24AM -0400, Jeff King wrote:
>> On Thu, Jun 09, 2011 at 06:59:15AM +0200, Andi Kleen wrote:
>>
>> > >  - Potential impact to people who do not use Gcc 4.6 with profile feedback
>> > >    is not explained away well, except for "Doesn't seem to make any
>> > >    difference."
>> >
>> > I merely went by "there are no new warnings" (I assume that's the main
>> > motivation)
>>
>> On your compiler and settings, perhaps. With your patch I get:
>>
>>   usage.c: In function ‘die’:
>>   usage.c:70:1: error: ‘noreturn’ function does return [-Werror]
>
> Ok.  Hmm, all I can say it compiled here.
>
> Ok then we have to remove it. I didn't really like Junio's approach
> to only do it for a single file because that would break with LTO / link
> time optimization which requires declarations to match between
> translation units.

Junio's approach didn't do it for a single file, it disabled the
NORETURN mechanism all together, by having a Makefile-switch.

> BTW 4.6.2 or so will have the problem fixed.

If it's only in 4.6 through 4.6.2, then we probably don't even need a
Makefile-switch for junio's approach at all; just checking the GCC
version should be reliable enough, no?