Problems compiling with KCFLAGS="-frecord-gcc-switches"

Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

Hi Josh,

when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
I get the folllowing build warning/error.

make allmodconfig
KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"

arch/x86/kvm/emulate.o: warning:
	objtool: .GCC.command.line+0x0: special: can't find new instruction

Building a full image aborts with:

WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
	from the (unknown reference) (unknown)
	to the variable .GCC.command.line:kvm_fastop_exception
FATAL: The relocation at __ex_table+0x4c references
section ".GCC.command.line" which is not executable, IOW
the kernel will fault if it ever tries to
jump to it.  Something is seriously wrong
and should be fixed.
make[2]: *** [arch/x86/kvm/kvm.o] Error 1

Any idea what might cause this problem ?

Thanks,
Guenter

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> Hi Josh,
> 
> when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> I get the folllowing build warning/error.
> 
> make allmodconfig
> KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> 
> arch/x86/kvm/emulate.o: warning:
> 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> 
> Building a full image aborts with:
> 
> WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> 	from the (unknown reference) (unknown)
> 	to the variable .GCC.command.line:kvm_fastop_exception
> FATAL: The relocation at __ex_table+0x4c references
> section ".GCC.command.line" which is not executable, IOW
> the kernel will fault if it ever tries to
> jump to it.  Something is seriously wrong
> and should be fixed.
> make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> 
> Any idea what might cause this problem ?
> 

Here is another interesting problem, seen when building arm64 allmodconfig
-CONFIG_CPU_BIG_ENDIAN +CONFIG_EFI.

kallsyms failure:
	relative symbol value 0xffffff9008073000 out of range in relative mode

This is due to symbols such as

000000000000000e n __efistub_$d

in the symbol table. Those are not filtered out by kallsyms, resulting in a
relative "base" address of 0x0e, and all other symbols are out of range.
Those symbols are only generated in efi/libstubs.

Any idea what might be going on there, and how to fix it ?
An easy fix would be something like

-       else if (stype == 'N')
+       else if (toupper(stype) == 'N')

in kallsyms, but that doesn't seem like a clean solution to me.

Thanks,
Guenter

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Josh Poimboeuf]

On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> Hi Josh,
> 
> when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> I get the folllowing build warning/error.
> 
> make allmodconfig
> KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> 
> arch/x86/kvm/emulate.o: warning:
> 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> 
> Building a full image aborts with:
> 
> WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> 	from the (unknown reference) (unknown)
> 	to the variable .GCC.command.line:kvm_fastop_exception
> FATAL: The relocation at __ex_table+0x4c references
> section ".GCC.command.line" which is not executable, IOW
> the kernel will fault if it ever tries to
> jump to it.  Something is seriously wrong
> and should be fixed.
> make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> 
> Any idea what might cause this problem ?

I think this is a GCC bug with the -frecord-gcc-switches feature.  Due
to some ambiguity in some inline asm in emulate.c, it's placing some
kernel code in the .GCC.command.line section.

This seems to fix it:


diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index a36254cbf776..d90cdc77e077 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -425,8 +425,10 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
 	#op " %al \n\t" \
 	FOP_RET
 
-asm(".global kvm_fastop_exception \n"
-    "kvm_fastop_exception: xor %esi, %esi; ret");
+asm(".pushsection .fixup, \"ax\"\n"
+    ".global kvm_fastop_exception \n"
+    "kvm_fastop_exception: xor %esi, %esi; ret\n"
+    ".popsection");
 
 FOP_START(setcc)
 FOP_SETCC(seto)

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Josh Poimboeuf]

On Fri, Sep 29, 2017 at 01:38:43PM -0700, Guenter Roeck wrote:
> On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> > Hi Josh,
> > 
> > when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> > I get the folllowing build warning/error.
> > 
> > make allmodconfig
> > KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> > ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> > 
> > arch/x86/kvm/emulate.o: warning:
> > 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> > 
> > Building a full image aborts with:
> > 
> > WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> > 	from the (unknown reference) (unknown)
> > 	to the variable .GCC.command.line:kvm_fastop_exception
> > FATAL: The relocation at __ex_table+0x4c references
> > section ".GCC.command.line" which is not executable, IOW
> > the kernel will fault if it ever tries to
> > jump to it.  Something is seriously wrong
> > and should be fixed.
> > make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> > 
> > Any idea what might cause this problem ?
> > 
> 
> Here is another interesting problem, seen when building arm64 allmodconfig
> -CONFIG_CPU_BIG_ENDIAN +CONFIG_EFI.
> 
> kallsyms failure:
> 	relative symbol value 0xffffff9008073000 out of range in relative mode
> 
> This is due to symbols such as
> 
> 000000000000000e n __efistub_$d
> 
> in the symbol table. Those are not filtered out by kallsyms, resulting in a
> relative "base" address of 0x0e, and all other symbols are out of range.
> Those symbols are only generated in efi/libstubs.
> 
> Any idea what might be going on there, and how to fix it ?
> An easy fix would be something like
> 
> -       else if (stype == 'N')
> +       else if (toupper(stype) == 'N')
> 
> in kallsyms, but that doesn't seem like a clean solution to me.

Not sure... What does 'n' mean?  I don't see it in the man page for nm.

-- 
Josh

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

On 09/29/2017 01:58 PM, Josh Poimboeuf wrote:
> On Fri, Sep 29, 2017 at 01:38:43PM -0700, Guenter Roeck wrote:
>> On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
>>> Hi Josh,
>>>
>>> when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
>>> I get the folllowing build warning/error.
>>>
>>> make allmodconfig
>>> KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
>>> ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
>>>
>>> arch/x86/kvm/emulate.o: warning:
>>> 	objtool: .GCC.command.line+0x0: special: can't find new instruction
>>>
>>> Building a full image aborts with:
>>>
>>> WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
>>> 	from the (unknown reference) (unknown)
>>> 	to the variable .GCC.command.line:kvm_fastop_exception
>>> FATAL: The relocation at __ex_table+0x4c references
>>> section ".GCC.command.line" which is not executable, IOW
>>> the kernel will fault if it ever tries to
>>> jump to it.  Something is seriously wrong
>>> and should be fixed.
>>> make[2]: *** [arch/x86/kvm/kvm.o] Error 1
>>>
>>> Any idea what might cause this problem ?
>>>
>>
>> Here is another interesting problem, seen when building arm64 allmodconfig
>> -CONFIG_CPU_BIG_ENDIAN +CONFIG_EFI.
>>
>> kallsyms failure:
>> 	relative symbol value 0xffffff9008073000 out of range in relative mode
>>
>> This is due to symbols such as
>>
>> 000000000000000e n __efistub_$d
>>
>> in the symbol table. Those are not filtered out by kallsyms, resulting in a
>> relative "base" address of 0x0e, and all other symbols are out of range.
>> Those symbols are only generated in efi/libstubs.
>>
>> Any idea what might be going on there, and how to fix it ?
>> An easy fix would be something like
>>
>> -       else if (stype == 'N')
>> +       else if (toupper(stype) == 'N')
>>
>> in kallsyms, but that doesn't seem like a clean solution to me.
> 
> Not sure... What does 'n' mean?  I don't see it in the man page for nm.
> 

Good question. As far as I can see, gcc uses it to mark symbols associated
with -frecord-gcc-switches, presumably in the .GCC.command.line section.

Guenter

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

On 09/29/2017 01:46 PM, Josh Poimboeuf wrote:
> On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
>> Hi Josh,
>>
>> when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
>> I get the folllowing build warning/error.
>>
>> make allmodconfig
>> KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
>> ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
>>
>> arch/x86/kvm/emulate.o: warning:
>> 	objtool: .GCC.command.line+0x0: special: can't find new instruction
>>
>> Building a full image aborts with:
>>
>> WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
>> 	from the (unknown reference) (unknown)
>> 	to the variable .GCC.command.line:kvm_fastop_exception
>> FATAL: The relocation at __ex_table+0x4c references
>> section ".GCC.command.line" which is not executable, IOW
>> the kernel will fault if it ever tries to
>> jump to it.  Something is seriously wrong
>> and should be fixed.
>> make[2]: *** [arch/x86/kvm/kvm.o] Error 1
>>
>> Any idea what might cause this problem ?
> 
> I think this is a GCC bug with the -frecord-gcc-switches feature.  Due
> to some ambiguity in some inline asm in emulate.c, it's placing some
> kernel code in the .GCC.command.line section.
> 

Interesting. Makes me wonder if it is doing something similar in the efi code.

> This seems to fix it:
> 
> 
> diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
> index a36254cbf776..d90cdc77e077 100644
> --- a/arch/x86/kvm/emulate.c
> +++ b/arch/x86/kvm/emulate.c
> @@ -425,8 +425,10 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
>   	#op " %al \n\t" \
>   	FOP_RET
>   
> -asm(".global kvm_fastop_exception \n"
> -    "kvm_fastop_exception: xor %esi, %esi; ret");
> +asm(".pushsection .fixup, \"ax\"\n"
> +    ".global kvm_fastop_exception \n"
> +    "kvm_fastop_exception: xor %esi, %esi; ret\n"
> +    ".popsection");
>   
>   FOP_START(setcc)
>   FOP_SETCC(seto)
> 

Yes, that fixes the problem for me.

Do you want to send a real patch ? If yes, feel free to add

Tested-by: Guenter Roeck <linux@roeck-us.net>

Thanks,
Guenter

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

Hi Josh,

On Fri, Sep 29, 2017 at 03:46:09PM -0500, Josh Poimboeuf wrote:
> On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> > Hi Josh,
> > 
> > when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> > I get the folllowing build warning/error.
> > 
> > make allmodconfig
> > KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> > ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> > 
> > arch/x86/kvm/emulate.o: warning:
> > 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> > 
> > Building a full image aborts with:
> > 
> > WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> > 	from the (unknown reference) (unknown)
> > 	to the variable .GCC.command.line:kvm_fastop_exception
> > FATAL: The relocation at __ex_table+0x4c references
> > section ".GCC.command.line" which is not executable, IOW
> > the kernel will fault if it ever tries to
> > jump to it.  Something is seriously wrong
> > and should be fixed.
> > make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> > 
> > Any idea what might cause this problem ?
> 
> I think this is a GCC bug with the -frecord-gcc-switches feature.  Due
> to some ambiguity in some inline asm in emulate.c, it's placing some
> kernel code in the .GCC.command.line section.
> 
> This seems to fix it:
> 
> 
> diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
> index a36254cbf776..d90cdc77e077 100644
> --- a/arch/x86/kvm/emulate.c
> +++ b/arch/x86/kvm/emulate.c
> @@ -425,8 +425,10 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
>  	#op " %al \n\t" \
>  	FOP_RET
>  
> -asm(".global kvm_fastop_exception \n"
> -    "kvm_fastop_exception: xor %esi, %esi; ret");
> +asm(".pushsection .fixup, \"ax\"\n"
> +    ".global kvm_fastop_exception \n"
> +    "kvm_fastop_exception: xor %esi, %esi; ret\n"
> +    ".popsection");
>  
>  FOP_START(setcc)
>  FOP_SETCC(seto)

Do you plan to submit a patch for this, or do you want me to do it ?

Thanks,
Guenter

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Josh Poimboeuf]

On Tue, Oct 03, 2017 at 10:54:57AM -0700, Guenter Roeck wrote:
> Hi Josh,
> 
> On Fri, Sep 29, 2017 at 03:46:09PM -0500, Josh Poimboeuf wrote:
> > On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> > > Hi Josh,
> > > 
> > > when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> > > I get the folllowing build warning/error.
> > > 
> > > make allmodconfig
> > > KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> > > ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> > > 
> > > arch/x86/kvm/emulate.o: warning:
> > > 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> > > 
> > > Building a full image aborts with:
> > > 
> > > WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> > > 	from the (unknown reference) (unknown)
> > > 	to the variable .GCC.command.line:kvm_fastop_exception
> > > FATAL: The relocation at __ex_table+0x4c references
> > > section ".GCC.command.line" which is not executable, IOW
> > > the kernel will fault if it ever tries to
> > > jump to it.  Something is seriously wrong
> > > and should be fixed.
> > > make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> > > 
> > > Any idea what might cause this problem ?
> > 
> > I think this is a GCC bug with the -frecord-gcc-switches feature.  Due
> > to some ambiguity in some inline asm in emulate.c, it's placing some
> > kernel code in the .GCC.command.line section.
> > 
> > This seems to fix it:
> > 
> > 
> > diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
> > index a36254cbf776..d90cdc77e077 100644
> > --- a/arch/x86/kvm/emulate.c
> > +++ b/arch/x86/kvm/emulate.c
> > @@ -425,8 +425,10 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
> >  	#op " %al \n\t" \
> >  	FOP_RET
> >  
> > -asm(".global kvm_fastop_exception \n"
> > -    "kvm_fastop_exception: xor %esi, %esi; ret");
> > +asm(".pushsection .fixup, \"ax\"\n"
> > +    ".global kvm_fastop_exception \n"
> > +    "kvm_fastop_exception: xor %esi, %esi; ret\n"
> > +    ".popsection");
> >  
> >  FOP_START(setcc)
> >  FOP_SETCC(seto)
> 
> Do you plan to submit a patch for this, or do you want me to do it ?

Sorry, I've been juggling a bunch of other bugs.  I'll submit it soon.

-- 
Josh

Re: Problems compiling with KCFLAGS="-frecord-gcc-switches"

[Guenter Roeck]

On Tue, Oct 03, 2017 at 01:00:23PM -0500, Josh Poimboeuf wrote:
> On Tue, Oct 03, 2017 at 10:54:57AM -0700, Guenter Roeck wrote:
> > Hi Josh,
> > 
> > On Fri, Sep 29, 2017 at 03:46:09PM -0500, Josh Poimboeuf wrote:
> > > On Fri, Sep 29, 2017 at 01:00:56PM -0700, Guenter Roeck wrote:
> > > > Hi Josh,
> > > > 
> > > > when trying to compile an image with KCFLAGS="-frecord-gcc-switches",
> > > > I get the folllowing build warning/error.
> > > > 
> > > > make allmodconfig
> > > > KCFLAGS="-frecord-gcc-switches" make arch/x86/kvm/emulate.o
> > > > ./tools/objtool/objtool check --no-unreachable "arch/x86/kvm/emulate.o"
> > > > 
> > > > arch/x86/kvm/emulate.o: warning:
> > > > 	objtool: .GCC.command.line+0x0: special: can't find new instruction
> > > > 
> > > > Building a full image aborts with:
> > > > 
> > > > WARNING: arch/x86/kvm/kvm.o(__ex_table+0x4c): Section mismatch in reference
> > > > 	from the (unknown reference) (unknown)
> > > > 	to the variable .GCC.command.line:kvm_fastop_exception
> > > > FATAL: The relocation at __ex_table+0x4c references
> > > > section ".GCC.command.line" which is not executable, IOW
> > > > the kernel will fault if it ever tries to
> > > > jump to it.  Something is seriously wrong
> > > > and should be fixed.
> > > > make[2]: *** [arch/x86/kvm/kvm.o] Error 1
> > > > 
> > > > Any idea what might cause this problem ?
> > > 
> > > I think this is a GCC bug with the -frecord-gcc-switches feature.  Due
> > > to some ambiguity in some inline asm in emulate.c, it's placing some
> > > kernel code in the .GCC.command.line section.
> > > 
> > > This seems to fix it:
> > > 
> > > 
> > > diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
> > > index a36254cbf776..d90cdc77e077 100644
> > > --- a/arch/x86/kvm/emulate.c
> > > +++ b/arch/x86/kvm/emulate.c
> > > @@ -425,8 +425,10 @@ static int fastop(struct x86_emulate_ctxt *ctxt, void (*fop)(struct fastop *));
> > >  	#op " %al \n\t" \
> > >  	FOP_RET
> > >  
> > > -asm(".global kvm_fastop_exception \n"
> > > -    "kvm_fastop_exception: xor %esi, %esi; ret");
> > > +asm(".pushsection .fixup, \"ax\"\n"
> > > +    ".global kvm_fastop_exception \n"
> > > +    "kvm_fastop_exception: xor %esi, %esi; ret\n"
> > > +    ".popsection");
> > >  
> > >  FOP_START(setcc)
> > >  FOP_SETCC(seto)
> > 
> > Do you plan to submit a patch for this, or do you want me to do it ?
> 
> Sorry, I've been juggling a bunch of other bugs.  I'll submit it soon.
> 
Welcome to the club ...

Thanks!
Guenter