Relocation problem with MIPS kernel modules

Relocation problem with MIPS kernel modules

[David VomLehn]

I have a MIPS loadable kernel module that, when I try to insmod it, causes the
kernel to emit the message:

	module xyz: dangerous relocation

This message appears in three different places in arch/mips/kernel/module.c,
but this one is coming from apply_r_mips_lo16_rel(). The module code at
the location at which the error message is generated appears to be pretty
bland:
	lw v0,28564(s1)
with the expected relocation type of R_MIPS_LO16. The relocation before it
is R_MIPS_HI16, as expected, but for a different symbol. Before *that*
is another R_MIPS_HI16 relocation entry for yet a third symbol.

According to the MIPS ABI, for what it's worth, "Each relocation type of
R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
it in the list of relocations." So, what's actually getting generated by
gcc and linker differs from the closest thing we have to an ABI of record for
MIPS processors.

What the kernel actually does differs from my interpretation of the MIPS ABI
in several particulars, which it has to do to relocate what it is actually
getting, but since I get the "dangerous relocation" error for a module that
appears to me to be valid, it may need to be fixed/enhanced.

So, who the heck knows what gcc and the linker are really generating and can
anyone suggest an algorithm for handling R_MIPS_HI16/R_MIPS_LO16 relocation
entries correctly?
--
David VomLehn

Re: Relocation problem with MIPS kernel modules

[David Daney]

David VomLehn wrote:
> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
> kernel to emit the message:
> 
> 	module xyz: dangerous relocation
> 
> This message appears in three different places in arch/mips/kernel/module.c,
> but this one is coming from apply_r_mips_lo16_rel(). The module code at
> the location at which the error message is generated appears to be pretty
> bland:
> 	lw v0,28564(s1)
> with the expected relocation type of R_MIPS_LO16. The relocation before it
> is R_MIPS_HI16, as expected, but for a different symbol. Before *that*
> is another R_MIPS_HI16 relocation entry for yet a third symbol.
> 
> According to the MIPS ABI, for what it's worth, "Each relocation type of
> R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
> it in the list of relocations." So, what's actually getting generated by
> gcc and linker differs from the closest thing we have to an ABI of record for
> MIPS processors.
> 
> What the kernel actually does differs from my interpretation of the MIPS ABI
> in several particulars, which it has to do to relocate what it is actually
> getting, but since I get the "dangerous relocation" error for a module that
> appears to me to be valid, it may need to be fixed/enhanced.
> 
> So, who the heck knows what gcc and the linker are really generating and can
> anyone suggest an algorithm for handling R_MIPS_HI16/R_MIPS_LO16 relocation
> entries correctly?

Without seeing your object module I am guessing.

Q: What version of binutils and GCC are you using?

It is the assembler's responsibility to properly sort the relocations. 
Some versions of binutils have bugs in this area.  If you are not using 
2.19.1, I suggest that you upgrade.

Unless you are compiling with -fsection-anchors there should be pairs of 
R_MIPS_HI16/R_MIPS_LO16 relocations.

David Daney

Re: Relocation problem with MIPS kernel modules

[David VomLehn]

On Thu, Jul 30, 2009 at 12:15:42PM -0700, David Daney wrote:
> David VomLehn wrote:
>> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
>> kernel to emit the message:
>>
>> 	module xyz: dangerous relocation
...
>> So, who the heck knows what gcc and the linker are really generating and can
>> anyone suggest an algorithm for handling R_MIPS_HI16/R_MIPS_LO16 relocation
>> entries correctly?
>
> Without seeing your object module I am guessing.
>
> Q: What version of binutils and GCC are you using?
>
> It is the assembler's responsibility to properly sort the relocations.  
> Some versions of binutils have bugs in this area.  If you are not using  
> 2.19.1, I suggest that you upgrade.
>
> Unless you are compiling with -fsection-anchors there should be pairs of  
> R_MIPS_HI16/R_MIPS_LO16 relocations.

We are downrev in our tool set; I'll try a newer version. Thanks!

> David Daney

David VomLehn

Re: Relocation problem with MIPS kernel modules

[Ralf Baechle]

On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:

> To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
> 	Linux MIPS Mailing List <linux-mips@linux-mips.org>
> Subject: Relocation problem with MIPS kernel modules
> Content-Type: text/plain; charset=us-ascii
> 
> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
> kernel to emit the message:
> 
> 	module xyz: dangerous relocation
> 
> This message appears in three different places in arch/mips/kernel/module.c,
> but this one is coming from apply_r_mips_lo16_rel(). The module code at

I'll change the messages to indicate the relocation type.

> the location at which the error message is generated appears to be pretty
> bland:
> 	lw v0,28564(s1)
> with the expected relocation type of R_MIPS_LO16. The relocation before it
> is R_MIPS_HI16, as expected, but for a different symbol. Before *that*
> is another R_MIPS_HI16 relocation entry for yet a third symbol.
> 
> According to the MIPS ABI, for what it's worth, "Each relocation type of
> R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
> it in the list of relocations." So, what's actually getting generated by
> gcc and linker differs from the closest thing we have to an ABI of record for
> MIPS processors.

The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
relocations of this sequence must use the same symbol, of course.  This is
a very old extension; I think it predates the Linux/MIPS port.

  Ralf

Re: Relocation problem with MIPS kernel modules

[David VomLehn]

On Mon, Aug 03, 2009 at 10:20:30AM +0100, Ralf Baechle wrote:
> On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:
> 
> > To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
> > 	Linux MIPS Mailing List <linux-mips@linux-mips.org>
> > Subject: Relocation problem with MIPS kernel modules
> > Content-Type: text/plain; charset=us-ascii
> > 
> > I have a MIPS loadable kernel module that, when I try to insmod it, causes the
> > kernel to emit the message:
> > 
> > 	module xyz: dangerous relocation
...
> > According to the MIPS ABI, for what it's worth, "Each relocation type of
> > R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
> > it in the list of relocations." So, what's actually getting generated by
> > gcc and linker differs from the closest thing we have to an ABI of record for
> > MIPS processors.
> 
> The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
> R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
> relocations of this sequence must use the same symbol, of course.  This is
> a very old extension; I think it predates the Linux/MIPS port.

Perhaps a foolish question, but is this documented anywhere? I know there is a
a document over at http://gcc.gnu.org/gcc-3.4/mips-abi.html addressing some
other MIPS ABI changes, but I didn't see this one. Obviously, we could put
documentation on the linux-mips Wiki, with pointers to other documents, but
I'm not sure this is the right place. I'm also concerned there could be other
ABI changes/extensions that need to be included.

>   Ralf

David

Re: Relocation problem with MIPS kernel modules

[David Daney]

David VomLehn wrote:
> On Mon, Aug 03, 2009 at 10:20:30AM +0100, Ralf Baechle wrote:
>> On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:
>>
>>> To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
>>> 	Linux MIPS Mailing List <linux-mips@linux-mips.org>
>>> Subject: Relocation problem with MIPS kernel modules
>>> Content-Type: text/plain; charset=us-ascii
>>>
>>> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
>>> kernel to emit the message:
>>>
>>> 	module xyz: dangerous relocation
> ...
>>> According to the MIPS ABI, for what it's worth, "Each relocation type of
>>> R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
>>> it in the list of relocations." So, what's actually getting generated by
>>> gcc and linker differs from the closest thing we have to an ABI of record for
>>> MIPS processors.
>> The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
>> R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
>> relocations of this sequence must use the same symbol, of course.  This is
>> a very old extension; I think it predates the Linux/MIPS port.
> 
> Perhaps a foolish question, but is this documented anywhere?

What more documentation do you need?  It's obvious if you read 
bfd/elf{32,64,xx}-mips.c :-).

David Daney

Re: Relocation problem with MIPS kernel modules

[Ralf Baechle]

On Mon, Aug 03, 2009 at 11:44:54AM -0700, David Daney wrote:

>>> R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
>>> relocations of this sequence must use the same symbol, of course.  This is
>>> a very old extension; I think it predates the Linux/MIPS port.
>>
>> Perhaps a foolish question, but is this documented anywhere?
>
> What more documentation do you need?  It's obvious if you read  
> bfd/elf{32,64,xx}-mips.c :-).

David ist (unforutunately ...) right.  I don't think this is documented
anywhere.  I also only learned it from reading the binutils sources.

  Ralf

Re: Relocation problem with MIPS kernel modules

[David Daney]

Ralf Baechle wrote:
> On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:
> 
>> To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
>> 	Linux MIPS Mailing List <linux-mips@linux-mips.org>
>> Subject: Relocation problem with MIPS kernel modules
>> Content-Type: text/plain; charset=us-ascii
>>
>> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
>> kernel to emit the message:
>>
>> 	module xyz: dangerous relocation
>>
>> This message appears in three different places in arch/mips/kernel/module.c,
>> but this one is coming from apply_r_mips_lo16_rel(). The module code at
> 
> I'll change the messages to indicate the relocation type.
> 
>> the location at which the error message is generated appears to be pretty
>> bland:
>> 	lw v0,28564(s1)
>> with the expected relocation type of R_MIPS_LO16. The relocation before it
>> is R_MIPS_HI16, as expected, but for a different symbol. Before *that*
>> is another R_MIPS_HI16 relocation entry for yet a third symbol.
>>
>> According to the MIPS ABI, for what it's worth, "Each relocation type of
>> R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
>> it in the list of relocations." So, what's actually getting generated by
>> gcc and linker differs from the closest thing we have to an ABI of record for
>> MIPS processors.
> 
> The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
> R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
> relocations of this sequence must use the same symbol, of course.  This is
> a very old extension; I think it predates the Linux/MIPS port.
> 

Actually I think it is the opposite:

RELOCATION RECORDS FOR [.text]:
OFFSET   TYPE              VALUE
00000000 R_MIPS_HI16       .bss+0x00000004
00000008 R_MIPS_LO16       .bss+0x00000004
00000014 R_MIPS_LO16       .bss+0x00000004

We load the hi16 value into a register and then use multiple lo16 
offsets for the follow loads and stores to the same location.  On a 
read-modify-write we only want to load the base address one time.

David Daney

Re: Relocation problem with MIPS kernel modules

[David VomLehn]

On Mon, Aug 03, 2009 at 12:33:25PM -0700, David Daney wrote:
> Ralf Baechle wrote:
>> On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:
>>
>>> To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
>>> 	Linux MIPS Mailing List <linux-mips@linux-mips.org>
>>> Subject: Relocation problem with MIPS kernel modules
>>> Content-Type: text/plain; charset=us-ascii
>>>
>>> I have a MIPS loadable kernel module that, when I try to insmod it, causes the
>>> kernel to emit the message:
>>>
>>> 	module xyz: dangerous relocation
...
>>
>> The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
>> R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
>> relocations of this sequence must use the same symbol, of course.  This is
>> a very old extension; I think it predates the Linux/MIPS port.
>>
>
> Actually I think it is the opposite:
>
> RELOCATION RECORDS FOR [.text]:
> OFFSET   TYPE              VALUE
> 00000000 R_MIPS_HI16       .bss+0x00000004
> 00000008 R_MIPS_LO16       .bss+0x00000004
> 00000014 R_MIPS_LO16       .bss+0x00000004
>
> We load the hi16 value into a register and then use multiple lo16  
> offsets for the follow loads and stores to the same location.  On a  
> read-modify-write we only want to load the base address one time.

This particular case is covered by the old MIPS Processor psABI:

	R_MIPS_LO16 entries without an R_MIPS_HI16 entry immediately preceding
	are orphaned and the previously defined R_MIPS_HI16 is used for
	computing the addend.

The code in module.c looks like it implements the extension to which Ralf
refers.

> David Daney

David VomLehn

Re: Relocation problem with MIPS kernel modules

[Ralf Baechle]

On Mon, Aug 03, 2009 at 01:15:21PM -0700, David VomLehn wrote:

> > Actually I think it is the opposite:
> >
> > RELOCATION RECORDS FOR [.text]:
> > OFFSET   TYPE              VALUE
> > 00000000 R_MIPS_HI16       .bss+0x00000004
> > 00000008 R_MIPS_LO16       .bss+0x00000004
> > 00000014 R_MIPS_LO16       .bss+0x00000004
> >
> > We load the hi16 value into a register and then use multiple lo16  
> > offsets for the follow loads and stores to the same location.  On a  
> > read-modify-write we only want to load the base address one time.
> 
> This particular case is covered by the old MIPS Processor psABI:
> 
> 	R_MIPS_LO16 entries without an R_MIPS_HI16 entry immediately preceding
> 	are orphaned and the previously defined R_MIPS_HI16 is used for
> 	computing the addend.
> 
> The code in module.c looks like it implements the extension to which Ralf
> refers.

Which is useful for for branch delay slot scheduling like:

	...
	j	1f
	lui	a0, %hi16(hello)
	...
	j	1f
	lui	a0, %hi16(hello)
	...
1:	jal	printf
	addiu	a0, %lo16(hello)

hello:	.asciz	"hello, hola\n"

The next and logical extension would be to permit multiple R_MIPS_LO16
records following a sequence of one or more R_MIPS_HI16 relocs as long as
all relate to the same symbol - which would be simple to support in the
kernel.

  Ralf

Re: Relocation problem with MIPS kernel modules

[David VomLehn]

On Tue, Aug 04, 2009 at 12:55:36AM +0100, Ralf Baechle wrote:
> On Mon, Aug 03, 2009 at 01:15:21PM -0700, David VomLehn wrote:
> 
> > > Actually I think it is the opposite:
> > >
> > > RELOCATION RECORDS FOR [.text]:
> > > OFFSET   TYPE              VALUE
> > > 00000000 R_MIPS_HI16       .bss+0x00000004
> > > 00000008 R_MIPS_LO16       .bss+0x00000004
> > > 00000014 R_MIPS_LO16       .bss+0x00000004
> > >
> > > We load the hi16 value into a register and then use multiple lo16  
> > > offsets for the follow loads and stores to the same location.  On a  
> > > read-modify-write we only want to load the base address one time.
> > 
> > This particular case is covered by the old MIPS Processor psABI:
> > 
> > 	R_MIPS_LO16 entries without an R_MIPS_HI16 entry immediately preceding
> > 	are orphaned and the previously defined R_MIPS_HI16 is used for
> > 	computing the addend.
> > 
> > The code in module.c looks like it implements the extension to which Ralf
> > refers.
> 
> Which is useful for for branch delay slot scheduling like:
> 
> 	...
> 	j	1f
> 	lui	a0, %hi16(hello)
> 	...
> 	j	1f
> 	lui	a0, %hi16(hello)
> 	...
> 1:	jal	printf
> 	addiu	a0, %lo16(hello)
> 
> hello:	.asciz	"hello, hola\n"
> 
> The next and logical extension would be to permit multiple R_MIPS_LO16
> records following a sequence of one or more R_MIPS_HI16 relocs as long as
> all relate to the same symbol - which would be simple to support in the
> kernel.

This is what the orphaned R_MIPS_LO16 entries mentioned in the psABI quote
are all about. The existing relocation code handles this in most cases, but
could be juiced up a bit to do the check to verify the symbols match between
the current R_MIPS_LO16 entry and the last R_MIPS_HI16 entry.

>   Ralf

David VomLehn

Re: Relocation problem with MIPS kernel modules

[Marshall B. Rogers]

@David VomLehn:

This is four days overdue, but here's a big document on the 32-bit MIPS ABI:
http://math-atlas.sourceforge.net/devel/assembly/mipsabi32.pdf

I haven't read it all, but it seems to cover everything you would ever
need, including the relocation types.

Regards,
Marshall

On Mon, Aug 3, 2009 at 8:19 PM, David VomLehn <dvomlehn@cisco.com> wrote:
>
> On Mon, Aug 03, 2009 at 10:20:30AM +0100, Ralf Baechle wrote:
> > On Thu, Jul 30, 2009 at 11:49:23AM -0700, David VomLehn wrote:
> >
> > > To: GCC Help Mailing List <gcc-help@gcc.gnu.org>,
> > >     Linux MIPS Mailing List <linux-mips@linux-mips.org>
> > > Subject: Relocation problem with MIPS kernel modules
> > > Content-Type: text/plain; charset=us-ascii
> > >
> > > I have a MIPS loadable kernel module that, when I try to insmod it, causes the
> > > kernel to emit the message:
> > >
> > >     module xyz: dangerous relocation
> ...
> > > According to the MIPS ABI, for what it's worth, "Each relocation type of
> > > R_MIPS_HI16 must have an associated R_MIPS_LO16 entry immediately following
> > > it in the list of relocations." So, what's actually getting generated by
> > > gcc and linker differs from the closest thing we have to an ABI of record for
> > > MIPS processors.
> >
> > The GNU tools as an extension over the MIPS ABI allows an arbitrary number of
> > R_MIPS_HI16 relocations to be followed by a R_MIPS_LO16 symbol.  All
> > relocations of this sequence must use the same symbol, of course.  This is
> > a very old extension; I think it predates the Linux/MIPS port.
>
> Perhaps a foolish question, but is this documented anywhere? I know there is a
> a document over at http://gcc.gnu.org/gcc-3.4/mips-abi.html addressing some
> other MIPS ABI changes, but I didn't see this one. Obviously, we could put
> documentation on the linux-mips Wiki, with pointers to other documents, but
> I'm not sure this is the right place. I'm also concerned there could be other
> ABI changes/extensions that need to be included.
>
> >   Ralf
>
> David

Re: Relocation problem with MIPS kernel modules

[Maciej W. Rozycki]

On Mon, 3 Aug 2009, David VomLehn wrote:

> > The next and logical extension would be to permit multiple R_MIPS_LO16
> > records following a sequence of one or more R_MIPS_HI16 relocs as long as
> > all relate to the same symbol - which would be simple to support in the
> > kernel.
> 
> This is what the orphaned R_MIPS_LO16 entries mentioned in the psABI quote
> are all about. The existing relocation code handles this in most cases, but
> could be juiced up a bit to do the check to verify the symbols match between
> the current R_MIPS_LO16 entry and the last R_MIPS_HI16 entry.

 Note there is no need to implement Ralf's suggestion -- if multiple HI16 
and LO16 relocations referring to the same symbol and addend are scattered 
throughout an object module, then the tools should combine them into pairs 
appropriately -- it does not matter exactly which ones of each are paired.  
Now once this has been done, a number of orphaned HI16 or LO16 relocations 
may remain, but there will only ever be one kind of these left and they 
can be attached to any of the pairs previously created.  My understanding 
is this is exactly what BFD does (modulo any possible bugs of course, as 
usually).

  Maciej