From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754251AbdLNXKz (ORCPT ); Thu, 14 Dec 2017 18:10:55 -0500 Received: from mail-oi0-f65.google.com ([209.85.218.65]:45985 "EHLO mail-oi0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754171AbdLNXKv (ORCPT ); Thu, 14 Dec 2017 18:10:51 -0500 X-Google-Smtp-Source: ACJfBovg3JSo8ZmkQn0BL5PNf4Ui/nkz2+zBMIAOchAvIj//6kqssvGIP6cBbrYmVaxOJfCaIXKKQ1lHXiCpjkxn0KE= MIME-Version: 1.0 In-Reply-To: References: <20171213092550.2774-1-mhocko@kernel.org> <20171213093110.3550-1-mhocko@kernel.org> <20171213093110.3550-2-mhocko@kernel.org>

From: Jann Horn Date: Fri, 15 Dec 2017 00:10:29 +0100 Message-ID: Subject: Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation To: John Hubbard Cc: Michal Hocko , Michael Kerrisk , Linux API , Khalid Aziz , Michael Ellerman , Andrew Morton , Russell King - ARM Linux , Andrea Arcangeli , linux-mm@kvack.org, LKML , linux-arch , Florian Weimer , Matthew Wilcox , Mike Rapoport , Cyril Hrubis , Pavel Machek , Michal Hocko Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Dec 15, 2017 at 12:06 AM, John Hubbard wrote: > On 12/13/2017 06:52 PM, Jann Horn wrote: >> On Wed, Dec 13, 2017 at 10:31 AM, Michal Hocko wrote: >>> From: John Hubbard > [...] >>> +.IP >>> +Furthermore, this option is extremely hazardous (when used on its own), because >>> +it forcibly removes pre-existing mappings, making it easy for a multi-threaded >>> +process to corrupt its own address space. >> >> I think this is worded unfortunately. It is dangerous if used >> incorrectly, and it's a good tool when used correctly. >> >> [...] >>> +Thread B need not create a mapping directly; simply making a library call >>> +that, internally, uses >>> +.I dlopen(3) >>> +to load some other shared library, will >>> +suffice. The dlopen(3) call will map the library into the process's address >>> +space. Furthermore, almost any library call may be implemented using this >>> +technique. >>> +Examples include brk(2), malloc(3), pthread_create(3), and the PAM libraries >>> +(http://www.linux-pam.org). >> >> This is arkward. This first mentions dlopen(), which is a very niche >> case, and then just very casually mentions the much bigger >> problem that tons of library functions can allocate memory through >> malloc(), causing mmap() calls, sometimes without that even being >> a documented property of the function. >> > > Hi Jann, > > Here is some proposed new wording, to address your two comments above. What do > you think of this: > > NOTE: this option can be hazardous (when used on its own), because it > forcibly removes pre-existing mappings, making it easy for a multi- > threaded process to corrupt its own address space. For example, thread A > looks through /proc//maps and locates an available address range, > while thread B simultaneously acquires part or all of that same address > range. Thread A then calls mmap(MAP_FIXED), effectively overwriting the > mapping that thread B created. > > Thread B need not create a mapping directly; simply making a library call > whose implementation calls malloc(3), mmap(), or dlopen(3) will suffice, > because those calls all create new mappings. Thanks! That sounds better to me. >>> +.IP >>> +Newer kernels >>> +(Linux 4.16 and later) have a >>> +.B MAP_FIXED_SAFE >>> +option that avoids the corruption problem; if available, MAP_FIXED_SAFE >>> +should be preferred over MAP_FIXED. >> >> This is bad advice. MAP_FIXED is completely safe if you use it on an address >> range you've allocated, and it is used in this way by core system libraries to >> place multiple VMAs in virtually contiguous memory, for example: > [...] >> MAP_FIXED is a better solution for these usecases than MAP_FIXED_SAFE, >> or whatever it ends up being called. Please remove this advice or, better, >> clarify what MAP_FIXED should be used for (creation of virtually contiguous >> VMAs) and what MAP_FIXED_SAFE should be used for (attempting to >> allocate memory at a fixed address for some reason, with a failure instead of >> the normal fallback to using a different address). >> > > Rather than risk another back-and-forth with Michal (who doesn't want any advice > on how to use this safely, in the man page), I've simply removed this advice > entirely. Makes sense. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jann Horn Subject: Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation Date: Fri, 15 Dec 2017 00:10:29 +0100 Message-ID: References: <20171213092550.2774-1-mhocko@kernel.org> <20171213093110.3550-1-mhocko@kernel.org> <20171213093110.3550-2-mhocko@kernel.org>

Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: In-Reply-To: Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: John Hubbard Cc: Michal Hocko , Michael Kerrisk , Linux API , Khalid Aziz , Michael Ellerman , Andrew Morton , Russell King - ARM Linux , Andrea Arcangeli , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, LKML , linux-arch , Florian Weimer , Matthew Wilcox , Mike Rapoport , Cyril Hrubis , Pavel Machek , Michal Hocko List-Id: linux-api@vger.kernel.org On Fri, Dec 15, 2017 at 12:06 AM, John Hubbard wrote: > On 12/13/2017 06:52 PM, Jann Horn wrote: >> On Wed, Dec 13, 2017 at 10:31 AM, Michal Hocko wrote: >>> From: John Hubbard > [...] >>> +.IP >>> +Furthermore, this option is extremely hazardous (when used on its own), because >>> +it forcibly removes pre-existing mappings, making it easy for a multi-threaded >>> +process to corrupt its own address space. >> >> I think this is worded unfortunately. It is dangerous if used >> incorrectly, and it's a good tool when used correctly. >> >> [...] >>> +Thread B need not create a mapping directly; simply making a library call >>> +that, internally, uses >>> +.I dlopen(3) >>> +to load some other shared library, will >>> +suffice. The dlopen(3) call will map the library into the process's address >>> +space. Furthermore, almost any library call may be implemented using this >>> +technique. >>> +Examples include brk(2), malloc(3), pthread_create(3), and the PAM libraries >>> +(http://www.linux-pam.org). >> >> This is arkward. This first mentions dlopen(), which is a very niche >> case, and then just very casually mentions the much bigger >> problem that tons of library functions can allocate memory through >> malloc(), causing mmap() calls, sometimes without that even being >> a documented property of the function. >> > > Hi Jann, > > Here is some proposed new wording, to address your two comments above. What do > you think of this: > > NOTE: this option can be hazardous (when used on its own), because it > forcibly removes pre-existing mappings, making it easy for a multi- > threaded process to corrupt its own address space. For example, thread A > looks through /proc//maps and locates an available address range, > while thread B simultaneously acquires part or all of that same address > range. Thread A then calls mmap(MAP_FIXED), effectively overwriting the > mapping that thread B created. > > Thread B need not create a mapping directly; simply making a library call > whose implementation calls malloc(3), mmap(), or dlopen(3) will suffice, > because those calls all create new mappings. Thanks! That sounds better to me. >>> +.IP >>> +Newer kernels >>> +(Linux 4.16 and later) have a >>> +.B MAP_FIXED_SAFE >>> +option that avoids the corruption problem; if available, MAP_FIXED_SAFE >>> +should be preferred over MAP_FIXED. >> >> This is bad advice. MAP_FIXED is completely safe if you use it on an address >> range you've allocated, and it is used in this way by core system libraries to >> place multiple VMAs in virtually contiguous memory, for example: > [...] >> MAP_FIXED is a better solution for these usecases than MAP_FIXED_SAFE, >> or whatever it ends up being called. Please remove this advice or, better, >> clarify what MAP_FIXED should be used for (creation of virtually contiguous >> VMAs) and what MAP_FIXED_SAFE should be used for (attempting to >> allocate memory at a fixed address for some reason, with a failure instead of >> the normal fallback to using a different address). >> > > Rather than risk another back-and-forth with Michal (who doesn't want any advice > on how to use this safely, in the man page), I've simply removed this advice > entirely. Makes sense. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot0-f197.google.com (mail-ot0-f197.google.com [74.125.82.197]) by kanga.kvack.org (Postfix) with ESMTP id 166C56B0253 for ; Thu, 14 Dec 2017 18:10:52 -0500 (EST) Received: by mail-ot0-f197.google.com with SMTP id 105so3767166oth.22 for ; Thu, 14 Dec 2017 15:10:52 -0800 (PST) Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v185sor1867137oia.45.2017.12.14.15.10.50 for (Google Transport Security); Thu, 14 Dec 2017 15:10:51 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: <20171213092550.2774-1-mhocko@kernel.org> <20171213093110.3550-1-mhocko@kernel.org> <20171213093110.3550-2-mhocko@kernel.org>

From: Jann Horn Date: Fri, 15 Dec 2017 00:10:29 +0100 Message-ID: Subject: Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation Content-Type: text/plain; charset="UTF-8" Sender: owner-linux-mm@kvack.org List-ID: To: John Hubbard Cc: Michal Hocko , Michael Kerrisk , Linux API , Khalid Aziz , Michael Ellerman , Andrew Morton , Russell King - ARM Linux , Andrea Arcangeli , linux-mm@kvack.org, LKML , linux-arch , Florian Weimer , Matthew Wilcox , Mike Rapoport , Cyril Hrubis , Pavel Machek , Michal Hocko On Fri, Dec 15, 2017 at 12:06 AM, John Hubbard wrote: > On 12/13/2017 06:52 PM, Jann Horn wrote: >> On Wed, Dec 13, 2017 at 10:31 AM, Michal Hocko wrote: >>> From: John Hubbard > [...] >>> +.IP >>> +Furthermore, this option is extremely hazardous (when used on its own), because >>> +it forcibly removes pre-existing mappings, making it easy for a multi-threaded >>> +process to corrupt its own address space. >> >> I think this is worded unfortunately. It is dangerous if used >> incorrectly, and it's a good tool when used correctly. >> >> [...] >>> +Thread B need not create a mapping directly; simply making a library call >>> +that, internally, uses >>> +.I dlopen(3) >>> +to load some other shared library, will >>> +suffice. The dlopen(3) call will map the library into the process's address >>> +space. Furthermore, almost any library call may be implemented using this >>> +technique. >>> +Examples include brk(2), malloc(3), pthread_create(3), and the PAM libraries >>> +(http://www.linux-pam.org). >> >> This is arkward. This first mentions dlopen(), which is a very niche >> case, and then just very casually mentions the much bigger >> problem that tons of library functions can allocate memory through >> malloc(), causing mmap() calls, sometimes without that even being >> a documented property of the function. >> > > Hi Jann, > > Here is some proposed new wording, to address your two comments above. What do > you think of this: > > NOTE: this option can be hazardous (when used on its own), because it > forcibly removes pre-existing mappings, making it easy for a multi- > threaded process to corrupt its own address space. For example, thread A > looks through /proc//maps and locates an available address range, > while thread B simultaneously acquires part or all of that same address > range. Thread A then calls mmap(MAP_FIXED), effectively overwriting the > mapping that thread B created. > > Thread B need not create a mapping directly; simply making a library call > whose implementation calls malloc(3), mmap(), or dlopen(3) will suffice, > because those calls all create new mappings. Thanks! That sounds better to me. >>> +.IP >>> +Newer kernels >>> +(Linux 4.16 and later) have a >>> +.B MAP_FIXED_SAFE >>> +option that avoids the corruption problem; if available, MAP_FIXED_SAFE >>> +should be preferred over MAP_FIXED. >> >> This is bad advice. MAP_FIXED is completely safe if you use it on an address >> range you've allocated, and it is used in this way by core system libraries to >> place multiple VMAs in virtually contiguous memory, for example: > [...] >> MAP_FIXED is a better solution for these usecases than MAP_FIXED_SAFE, >> or whatever it ends up being called. Please remove this advice or, better, >> clarify what MAP_FIXED should be used for (creation of virtually contiguous >> VMAs) and what MAP_FIXED_SAFE should be used for (attempting to >> allocate memory at a fixed address for some reason, with a failure instead of >> the normal fallback to using a different address). >> > > Rather than risk another back-and-forth with Michal (who doesn't want any advice > on how to use this safely, in the man page), I've simply removed this advice > entirely. Makes sense. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org