RFC: A new MIPS64 ABI

RFC: A new MIPS64 ABI

[David Daney]

Background:

Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
user virtual memory space.  This is due the way MIPS32 memory space is
segmented.  Only the range from 0..2^31-1 is available.  Pointer
values are always sign extended.

Because there are not already enough MIPS ABIs, I present the ...

Proposal: A new ABI to support 4GB of address space with 32-bit
pointers.

The proposed new ABI would only be available on MIPS64 platforms.  It
would be identical to the current MIPS n32 ABI *except* that pointers
would be zero-extended rather than sign-extended when resident in
registers.  In the remainder of this document I will call it
'n32-big'.  As a result, applications would have access to a full 4GB
of virtual address space.  The operating environment would be
configured such that the entire lower 4GB of the virtual address space
was available to the program.


At a low level here is how it would work:

1) Load a pointer to a register from memory:

n32:
	LW $reg, offset($reg)

n32-big:
	LWU $reg, offset($reg)

2) Load an address constant into a register:

n32:
	LUI $reg, high_part
	ORI $reg, low_part

n32-big:
	ORI $reg, high_part
	DSLL $reg, $reg, 16
	ORI $reg, low_part


Q: What would have to change to make this work?

o A new ELF header flag to denote the ABI.

o Linker support to use proper library search paths, and linker scrips
   to set the INTERP program header, etc.

o GCC has to emit code for the new ABI.

o Could all existing n32 relocation types be used?  I think so.

o Runtime libraries would have to be placed in a new location
   (/lib32big, /usr/lib32big ...)

o The C library's ld.so would have to use a distinct LD_LIBRARY_PATH
   for n32-big code.

o What would the Linux system call interface be?  I would propose
   using the existing Linux n32 system call interface.  Most system
   calls would just work.  Some, that pass pointers in in-memory
   structures, might require kernel modifications (sigaction() for
   example).

Re: RFC: A new MIPS64 ABI

[Alexandre Oliva]

On Feb 14, 2011, David Daney <ddaney@caviumnetworks.com> wrote:

> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space.  This is due the way MIPS32 memory space is
> segmented.  Only the range from 0..2^31-1 is available.  Pointer
> values are always sign extended.

> The proposed new ABI would only be available on MIPS64 platforms.  It
> would be identical to the current MIPS n32 ABI *except* that pointers
> would be zero-extended rather than sign-extended when resident in
> registers.

FTR, I don't really know why my Yeeloong is limited to 31-bit addresses,
and I kind of hoped an n32 userland would improve that WRT o32, without
wasting memory with longer pointers like n64 would.

So, sorry if this is a dumb question, but wouldn't it be much easier to
keep on using sign-extended addresses, and just make sure the kernel
never allocates a virtual memory range that crosses a sign-bit change,
or whatever other reason there is for addresses to be limited to the
positive 2GB range in n32?

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:
>
>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>> user virtual memory space.  This is due the way MIPS32 memory space is
>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>> values are always sign extended.
>
>> The proposed new ABI would only be available on MIPS64 platforms.  It
>> would be identical to the current MIPS n32 ABI *except* that pointers
>> would be zero-extended rather than sign-extended when resident in
>> registers.
>
> FTR, I don't really know why my Yeeloong is limited to 31-bit addresses,
> and I kind of hoped an n32 userland would improve that WRT o32, without
> wasting memory with longer pointers like n64 would.
>
> So, sorry if this is a dumb question, but wouldn't it be much easier to
> keep on using sign-extended addresses, and just make sure the kernel
> never allocates a virtual memory range that crosses a sign-bit change,
> or whatever other reason there is for addresses to be limited to the
> positive 2GB range in n32?
>

No, it is not possible.  The MIPS (and MIPS64) hardware architecture 
does not allow userspace access to addresses with the high bit (two bits 
for mips64) set.

Your complaint is a good summary of why I am thinking about n32-big.

David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 12:29 PM, David Daney wrote:
> Background:
>
> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space. This is due the way MIPS32 memory space is
> segmented. Only the range from 0..2^31-1 is available. Pointer
> values are always sign extended.
>
> Because there are not already enough MIPS ABIs, I present the ...
>
> Proposal: A new ABI to support 4GB of address space with 32-bit
> pointers.
>
> The proposed new ABI would only be available on MIPS64 platforms. It
> would be identical to the current MIPS n32 ABI *except* that pointers
> would be zero-extended rather than sign-extended when resident in
> registers. In the remainder of this document I will call it
> 'n32-big'. As a result, applications would have access to a full 4GB
> of virtual address space. The operating environment would be
> configured such that the entire lower 4GB of the virtual address space
> was available to the program.
>
>
> At a low level here is how it would work:
>
> 1) Load a pointer to a register from memory:
>
> n32:
> LW $reg, offset($reg)
>
> n32-big:
> LWU $reg, offset($reg)
>
> 2) Load an address constant into a register:
>
> n32:
> LUI $reg, high_part
> ORI $reg, low_part

That is not reality.  Really it is:

LUI $reg, R_MIPS_HI16
ADDIU $reg, R_MIPS_LO16


>
> n32-big:
> ORI $reg, high_part
> DSLL $reg, $reg, 16
> ORI $reg, low_part
>

This one would really be:

ORI $reg, R_MIPS_HI16
DSLL $reg, $reg, 16
ADDIU $reg, R_MIPS_LO16


>
> Q: What would have to change to make this work?
>
> o A new ELF header flag to denote the ABI.
>
> o Linker support to use proper library search paths, and linker scrips
> to set the INTERP program header, etc.
>
> o GCC has to emit code for the new ABI.
>
> o Could all existing n32 relocation types be used? I think so.
>
> o Runtime libraries would have to be placed in a new location
> (/lib32big, /usr/lib32big ...)
>
> o The C library's ld.so would have to use a distinct LD_LIBRARY_PATH
> for n32-big code.
>
> o What would the Linux system call interface be? I would propose
> using the existing Linux n32 system call interface. Most system
> calls would just work. Some, that pass pointers in in-memory
> structures, might require kernel modifications (sigaction() for
> example).
>

Re: RFC: A new MIPS64 ABI

[Alexandre Oliva]

On Feb 15, 2011, David Daney <ddaney@caviumnetworks.com> wrote:

> On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
>> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:

>> So, sorry if this is a dumb question, but wouldn't it be much easier to
>> keep on using sign-extended addresses, and just make sure the kernel
>> never allocates a virtual memory range that crosses a sign-bit change,

> No, it is not possible.  The MIPS (and MIPS64) hardware architecture
> does not allow userspace access to addresses with the high bit (two
> bits for mips64) set.

Interesting.  I guess this makes it far easier to transition to the u32
ABI: n32 addresses all have the 32-bit MSB bit clear, so n32 binaries
can be used within u32 environments, as long as the environment refrains
from using addresses that have the MSB bit set.

So we could switch lib32 to u32, have a machine-specific bit set for u32
binaries, and if the kernel starts an executable or interpreter that has
that bit clear, it will refrain from allocating any n32-invalid address
for that process.  Furthermore, libc, upon loading a library, should be
able to notify the kernel when an n32 library is to be loaded, to which
the kernel would respond either with failure (if that process already
uses u32-valid but n32-invalid addresses) or success (switching to n32
mode if not in it already).

Am I missing any other issues?

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: RFC: A new MIPS64 ABI

[David Daney]

On 05/06/2011 01:29 AM, Alexandre Oliva wrote:
> On Feb 15, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:
>
>> On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
>>> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>   wrote:
>
>>> So, sorry if this is a dumb question, but wouldn't it be much easier to
>>> keep on using sign-extended addresses, and just make sure the kernel
>>> never allocates a virtual memory range that crosses a sign-bit change,
>
>> No, it is not possible.  The MIPS (and MIPS64) hardware architecture
>> does not allow userspace access to addresses with the high bit (two
>> bits for mips64) set.
>
> Interesting.  I guess this makes it far easier to transition to the u32
> ABI: n32 addresses all have the 32-bit MSB bit clear, so n32 binaries
> can be used within u32 environments, as long as the environment refrains
> from using addresses that have the MSB bit set.
>

Correct.

> So we could switch lib32 to u32, have a machine-specific bit set for u32
> binaries, and if the kernel starts an executable or interpreter that has
> that bit clear, it will refrain from allocating any n32-invalid address
> for that process.  Furthermore, libc, upon loading a library, should be
> able to notify the kernel when an n32 library is to be loaded, to which
> the kernel would respond either with failure (if that process already
> uses u32-valid but n32-invalid addresses) or success (switching to n32
> mode if not in it already).
>
> Am I missing any other issues?

No, this is pretty much what Ralf and I came up with on IRC.

We tag u32 objects (in a similar manner to how non-executable stack is 
done).  The linker will propagate the u32 tag as it links things together.

u32 shared libraries are compatible with legacy n32 binaries as long as 
the OS doesn't map any memory where the address has bit 31 set.

When the OS loads an n32 executable it would check the u32 tag (both of 
the executable and ld.so) and adjust its memory allocation strategy.

The OS will continue to map the VDSO at the 2GB point.  This will cause 
the maximum size of any object to be compatible with the 32-bit n32 
ptrdiff_t.

I think once the OS puts a process into u32 mode, there is no going 
back.  We would just have ld.so refuse to load any shared objects that 
were not compatible with the current mode.

We would continue to place libraries in /lib32, /usr/lib32, 
/usr/local/lib32, etc.


David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 05/09/2011 07:28 AM, Ralf Baechle wrote:
> On Mon, Feb 21, 2011 at 07:45:41PM +0000, Richard Sandiford wrote:
>
>> David Daney<ddaney@caviumnetworks.com>  writes:
>>> Background:
>>>
>>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>>> user virtual memory space.  This is due the way MIPS32 memory space is
>>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>>> values are always sign extended.
>>>
>>> Because there are not already enough MIPS ABIs, I present the ...
>>>
>>> Proposal: A new ABI to support 4GB of address space with 32-bit
>>> pointers.
>>
>> FWIW, I'd be happy to see this go into GCC.
>
> So am I for the kernel primarily because it's not really a new ABI but
> an enhancement of the existing N32 ABI.
>

The patches are resting peacefully on my laptop.  Now with endorcements 
like these, I may be forced to actually finish them...

David Daney

Re: RFC: A new MIPS64 ABI

[Ralf Baechle]

On Mon, Feb 21, 2011 at 07:45:41PM +0000, Richard Sandiford wrote:

> David Daney <ddaney@caviumnetworks.com> writes:
> > Background:
> >
> > Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> > user virtual memory space.  This is due the way MIPS32 memory space is
> > segmented.  Only the range from 0..2^31-1 is available.  Pointer
> > values are always sign extended.
> >
> > Because there are not already enough MIPS ABIs, I present the ...
> >
> > Proposal: A new ABI to support 4GB of address space with 32-bit
> > pointers.
> 
> FWIW, I'd be happy to see this go into GCC.

So am I for the kernel primarily because it's not really a new ABI but
an enhancement of the existing N32 ABI.

  Ralf

Re: RFC: A new MIPS64 ABI

[Richard Sandiford]

David Daney <ddaney@caviumnetworks.com> writes:
> Background:
>
> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space.  This is due the way MIPS32 memory space is
> segmented.  Only the range from 0..2^31-1 is available.  Pointer
> values are always sign extended.
>
> Because there are not already enough MIPS ABIs, I present the ...
>
> Proposal: A new ABI to support 4GB of address space with 32-bit
> pointers.

FWIW, I'd be happy to see this go into GCC.

Richard