[PATCH] mmap.2: MAP_FIXED is no longer discouraged

[PATCH] mmap.2: MAP_FIXED is no longer discouraged

[john.hubbard]

From: John Hubbard <jhubbard@nvidia.com>

MAP_FIXED has been widely used for a very long time, yet the man
page still claims that "the use of this option is discouraged".

The documentation assumes that "less portable" == "must be discouraged".

Instead of discouraging something that is so useful and widely used,
change the documentation to explain its limitations better.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
While reviewing Michal Hocko's man page update for MAP_FIXED_SAFE,
I noticed that MAP_FIXED was no longer reflecting the current
situation, so here is a patch to bring it into the year 2017.

 man2/mmap.2 | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/man2/mmap.2 b/man2/mmap.2
index 385f3bfd5..a5a8eb47a 100644
--- a/man2/mmap.2
+++ b/man2/mmap.2
@@ -222,8 +222,10 @@ part of the existing mapping(s) will be discarded.
 If the specified address cannot be used,
 .BR mmap ()
 will fail.
-Because requiring a fixed address for a mapping is less portable,
-the use of this option is discouraged.
+Software that aspires to be portable should use this option with care, keeping
+in mind that the exact layout of a process' memory map is allowed to change
+significantly between kernel versions, C library versions, and operating system
+releases.
 .TP
 .B MAP_GROWSDOWN
 This flag is used for stacks.
-- 
2.15.1

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

[Matthew Wilcox]

On Fri, Dec 01, 2017 at 06:16:26PM -0800, john.hubbard@gmail.com wrote:
> MAP_FIXED has been widely used for a very long time, yet the man
> page still claims that "the use of this option is discouraged".

I think we should continue to discourage the use of this option, but
I'm going to include some of your text in my replacement paragraph ...

-Because requiring a fixed address for a mapping is less portable,
-the use of this option is discouraged.
+The use of this option is discouraged because it forcibly unmaps any
+existing mapping at that address.  Programs which use this option need
+to be aware that their memory map may change significantly from one run to
+the next, depending on library versions, kernel versions and random numbers.
+In a threaded process, checking the existing mappings can race against
+a new dynamic library being loaded, for example in response to another
+thread making a library call which causes a PAM module to be loaded.

(I don't love this text, in particular "PAM module".  I'm going off to
use the ATM machine now.  Please edit.)

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

[Jann Horn]

On Sat, Dec 2, 2017 at 4:05 PM, Matthew Wilcox <willy@infradead.org> wrote:
> On Fri, Dec 01, 2017 at 06:16:26PM -0800, john.hubbard@gmail.com wrote:
>> MAP_FIXED has been widely used for a very long time, yet the man
>> page still claims that "the use of this option is discouraged".
>
> I think we should continue to discourage the use of this option, but
> I'm going to include some of your text in my replacement paragraph ...
>
> -Because requiring a fixed address for a mapping is less portable,
> -the use of this option is discouraged.
> +The use of this option is discouraged because it forcibly unmaps any
> +existing mapping at that address.  Programs which use this option need
> +to be aware that their memory map may change significantly from one run to
> +the next, depending on library versions, kernel versions and random numbers.

How about adding something explicit about when it's okay to use MAP_FIXED?
"This option should only be used to displace an existing mapping that is
controlled by the caller, or part of such a mapping." or something like that?

> +In a threaded process, checking the existing mappings can race against
> +a new dynamic library being loaded

malloc() and its various callers can also cause mmap() calls, which is probably
more relevant than library loading.

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

[Matthew Wilcox]

On Sat, Dec 02, 2017 at 07:49:20PM +0100, Jann Horn wrote:
> On Sat, Dec 2, 2017 at 4:05 PM, Matthew Wilcox <willy@infradead.org> wrote:
> > On Fri, Dec 01, 2017 at 06:16:26PM -0800, john.hubbard@gmail.com wrote:
> >> MAP_FIXED has been widely used for a very long time, yet the man
> >> page still claims that "the use of this option is discouraged".
> >
> > I think we should continue to discourage the use of this option, but
> > I'm going to include some of your text in my replacement paragraph ...
> >
> > -Because requiring a fixed address for a mapping is less portable,
> > -the use of this option is discouraged.
> > +The use of this option is discouraged because it forcibly unmaps any
> > +existing mapping at that address.  Programs which use this option need
> > +to be aware that their memory map may change significantly from one run to
> > +the next, depending on library versions, kernel versions and random numbers.
> 
> How about adding something explicit about when it's okay to use MAP_FIXED?
> "This option should only be used to displace an existing mapping that is
> controlled by the caller, or part of such a mapping." or something like that?
> 
> > +In a threaded process, checking the existing mappings can race against
> > +a new dynamic library being loaded
> 
> malloc() and its various callers can also cause mmap() calls, which is probably
> more relevant than library loading.

That's a bit more expected though.  "I called malloc and my address
space changed".  Well, yeah.  But "I called getpwnam and my address
space changed" is a bit more surprising.  Don't you think?

Maybe that should be up front rather than buried at the end of the sentence.

"In a multi-threaded process, the address space can change in response to
virtually any library call.  This is because almost any library call may be
implemented by using dlopen(3) to load another shared library, which will be
mapped into the process's address space.  The PAM libraries are an excellent
example, as well as more obvious examples like brk(2), malloc(3) and even
pthread_create(3)."

What do you think?

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

[John Hubbard]

On 12/02/2017 02:19 PM, Matthew Wilcox wrote:
> On Sat, Dec 02, 2017 at 07:49:20PM +0100, Jann Horn wrote:
>> On Sat, Dec 2, 2017 at 4:05 PM, Matthew Wilcox <willy@infradead.org> wrote:
>>> On Fri, Dec 01, 2017 at 06:16:26PM -0800, john.hubbard@gmail.com wrote:
>>>> MAP_FIXED has been widely used for a very long time, yet the man
>>>> page still claims that "the use of this option is discouraged".
>>>
>>> I think we should continue to discourage the use of this option, but
>>> I'm going to include some of your text in my replacement paragraph ...
>>>
>>> -Because requiring a fixed address for a mapping is less portable,
>>> -the use of this option is discouraged.
>>> +The use of this option is discouraged because it forcibly unmaps any
>>> +existing mapping at that address.  Programs which use this option need
>>> +to be aware that their memory map may change significantly from one run to
>>> +the next, depending on library versions, kernel versions and random numbers.
>>
>> How about adding something explicit about when it's okay to use MAP_FIXED?
>> "This option should only be used to displace an existing mapping that is
>> controlled by the caller, or part of such a mapping." or something like that?
>>
>>> +In a threaded process, checking the existing mappings can race against
>>> +a new dynamic library being loaded
>>
>> malloc() and its various callers can also cause mmap() calls, which is probably
>> more relevant than library loading.
> 
> That's a bit more expected though.  "I called malloc and my address
> space changed".  Well, yeah.  But "I called getpwnam and my address
> space changed" is a bit more surprising.  Don't you think?
> 
> Maybe that should be up front rather than buried at the end of the sentence.
> 
> "In a multi-threaded process, the address space can change in response to
> virtually any library call.  This is because almost any library call may be
> implemented by using dlopen(3) to load another shared library, which will be
> mapped into the process's address space.  The PAM libraries are an excellent
> example, as well as more obvious examples like brk(2), malloc(3) and even
> pthread_create(3)."
> 
> What do you think?
> 

I'm working on some updated wording to capture these points. I'm even slower
at writing than I am at coding, so there will be a somewhat-brief pause here... :)

thanks,
John Hubbard
NVIDIA

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

[John Hubbard]

On 12/02/2017 02:19 PM, Matthew Wilcox wrote:
> On Sat, Dec 02, 2017 at 07:49:20PM +0100, Jann Horn wrote:
>> On Sat, Dec 2, 2017 at 4:05 PM, Matthew Wilcox <willy@infradead.org> wrote:
>>> On Fri, Dec 01, 2017 at 06:16:26PM -0800, john.hubbard@gmail.com wrote:
[...]
> 
> Maybe that should be up front rather than buried at the end of the sentence.
> 
> "In a multi-threaded process, the address space can change in response to
> virtually any library call.  This is because almost any library call may be
> implemented by using dlopen(3) to load another shared library, which will be
> mapped into the process's address space.  The PAM libraries are an excellent
> example, as well as more obvious examples like brk(2), malloc(3) and even
> pthread_create(3)."
> 
> What do you think?
> 

Hi Matthew,

Here is a new version, based on your and Jann's comments. I also added a
reference to MAP_FIXED_SAFE. If it looks close, I'll send a v2 with proper
formatting applied.

I did wonder briefly if your ATM reference was a oblique commentary about
security, but then realized...you probably just needed some cash. :)

-----

This option is extremely hazardous (when used on its own) and moderately
non-portable.

On portability: a process's memory map may change significantly from one
run to the next, depending on library versions, kernel versions and ran‐
dom numbers.

On hazards: this option forcibly removes pre-existing  mappings,  making
it easy for a multi-threaded process to corrupt its own address space.

For  example,  thread  A  looks  through /proc/<pid>/maps and locates an
available address range, while thread B simultaneously acquires part  or
all  of  that  same  address range. Thread A then calls mmap(MAP_FIXED),
effectively overwriting thread B's mapping.

Thread B need not create a mapping directly;  simply  making  a  library
call that, internally, uses dlopen(3) to load some other shared library,
will suffice. The dlopen(3) call will map the library into the process's
address  space.  Furthermore, almost any library call may be implemented
using this technique.  Examples include brk(2), malloc(3),  pthread_cre‐
ate(3), and the PAM libraries (http://www.linux-pam.org).

Given the above limitations, one of the very few ways to use this option
safely is: mmap() a region, without specifying MAP_FIXED.  Then,  within
that  region,  call  mmap(MAP_FIXED) to suballocate regions. This avoids
both the portability problem (because the first mmap call lets the  ker‐
nel pick the address), and the address space corruption problem (because
the region being overwritten is already owned by the calling thread).

Newer kernels (Linux 4.16 and later) have a MAP_FIXED_SAFE  option  that
avoids  the  corruption  problem; if available, MAP_FIXED_SAFE should be
preferred over MAP_FIXED.


thanks,
John Hubbard
NVIDIA