All of lore.kernel.org
 help / color / mirror / Atom feed
From: hpa@zytor.com
To: "Kirill A. Shutemov" <kirill@shutemov.name>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Cc: linux-arch <linux-arch@vger.kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Michal Hocko <mhocko@suse.com>,
	linux-kernel@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Andy Lutomirski <luto@amacapital.net>,
	Dave Hansen <dave.hansen@intel.com>,
	linux-mm@kvack.org, x86@kernel.org,
	Andi Kleen <ak@linux.intel.com>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH 26/26] x86/mm: allow to have userspace mappings above 47-bits
Date: Mon, 20 Mar 2017 11:08:41 -0700	[thread overview]
Message-ID: <95631D05-2CA2-4967-A29E-DB396C76F62D@zytor.com> (raw)
In-Reply-To: <CAFZ8GQx2JmEECQHEsKOymP8nDv9YHfLgcK80R75gM+r-1q-owQ@mail.gmail.com>

On March 19, 2017 1:26:58 AM PDT, "Kirill A. Shutemov" <kirill@shutemov.name> wrote:
>On Mar 19, 2017 09:25, "Aneesh Kumar K.V"
><aneesh.kumar@linux.vnet.ibm.com>
>wrote:
>
>"Kirill A. Shutemov" <kirill@shutemov.name> writes:
>
>> On Fri, Mar 17, 2017 at 11:23:54PM +0530, Aneesh Kumar K.V wrote:
>>> "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> writes:
>>>
>>> > On x86, 5-level paging enables 56-bit userspace virtual address
>space.
>>> > Not all user space is ready to handle wide addresses. It's known
>that
>>> > at least some JIT compilers use higher bits in pointers to encode
>their
>>> > information. It collides with valid pointers with 5-level paging
>and
>>> > leads to crashes.
>>> >
>>> > To mitigate this, we are not going to allocate virtual address
>space
>>> > above 47-bit by default.
>>> >
>>> > But userspace can ask for allocation from full address space by
>>> > specifying hint address (with or without MAP_FIXED) above 47-bits.
>>> >
>>> > If hint address set above 47-bit, but MAP_FIXED is not specified,
>we
>try
>>> > to look for unmapped area by specified address. If it's already
>>> > occupied, we look for unmapped area in *full* address space,
>rather
>than
>>> > from 47-bit window.
>>> >
>>> > This approach helps to easily make application's memory allocator
>aware
>>> > about large address space without manually tracking allocated
>virtual
>>> > address space.
>>> >
>>>
>>> So if I have done a successful mmap which returned > 128TB what
>should a
>>> following mmap(0,...) return ? Should that now search the *full*
>address
>>> space or below 128TB ?
>>
>> No, I don't think so. And this implementation doesn't do this.
>>
>> It's safer this way: if an library can't handle high addresses, it's
>> better not to switch it automagically to full address space if other
>part
>> of the process requested high address.
>>
>
>What is the epectation when the hint addr is below 128TB but addr + len
>>
>128TB ? Should such mmap request fail ?
>
>
>Yes, I believe so.

This *better* be conditional on some kind of settable limit.  Having a barrier in the middle of the address space for no apparent reason to "clean" software is insane.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

WARNING: multiple messages have this Message-ID (diff)
From: hpa@zytor.com
To: "Kirill A. Shutemov" <kirill@shutemov.name>,
	"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Cc: linux-arch <linux-arch@vger.kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Michal Hocko <mhocko@suse.com>,
	linux-kernel@vger.kernel.org,
	Andrew Morton <akpm@linux-foundation.org>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Arnd Bergmann <arnd@arndb.de>,
	Andy Lutomirski <luto@amacapital.net>,
	Dave Hansen <dave.hansen@intel.com>,
	linux-mm@kvack.org, x86@kernel.org,
	Andi Kleen <ak@linux.intel.com>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH 26/26] x86/mm: allow to have userspace mappings above 47-bits
Date: Mon, 20 Mar 2017 11:08:41 -0700	[thread overview]
Message-ID: <95631D05-2CA2-4967-A29E-DB396C76F62D@zytor.com> (raw)
In-Reply-To: <CAFZ8GQx2JmEECQHEsKOymP8nDv9YHfLgcK80R75gM+r-1q-owQ@mail.gmail.com>

On March 19, 2017 1:26:58 AM PDT, "Kirill A. Shutemov" <kirill@shutemov.name> wrote:
>On Mar 19, 2017 09:25, "Aneesh Kumar K.V"
><aneesh.kumar@linux.vnet.ibm.com>
>wrote:
>
>"Kirill A. Shutemov" <kirill@shutemov.name> writes:
>
>> On Fri, Mar 17, 2017 at 11:23:54PM +0530, Aneesh Kumar K.V wrote:
>>> "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> writes:
>>>
>>> > On x86, 5-level paging enables 56-bit userspace virtual address
>space.
>>> > Not all user space is ready to handle wide addresses. It's known
>that
>>> > at least some JIT compilers use higher bits in pointers to encode
>their
>>> > information. It collides with valid pointers with 5-level paging
>and
>>> > leads to crashes.
>>> >
>>> > To mitigate this, we are not going to allocate virtual address
>space
>>> > above 47-bit by default.
>>> >
>>> > But userspace can ask for allocation from full address space by
>>> > specifying hint address (with or without MAP_FIXED) above 47-bits.
>>> >
>>> > If hint address set above 47-bit, but MAP_FIXED is not specified,
>we
>try
>>> > to look for unmapped area by specified address. If it's already
>>> > occupied, we look for unmapped area in *full* address space,
>rather
>than
>>> > from 47-bit window.
>>> >
>>> > This approach helps to easily make application's memory allocator
>aware
>>> > about large address space without manually tracking allocated
>virtual
>>> > address space.
>>> >
>>>
>>> So if I have done a successful mmap which returned > 128TB what
>should a
>>> following mmap(0,...) return ? Should that now search the *full*
>address
>>> space or below 128TB ?
>>
>> No, I don't think so. And this implementation doesn't do this.
>>
>> It's safer this way: if an library can't handle high addresses, it's
>> better not to switch it automagically to full address space if other
>part
>> of the process requested high address.
>>
>
>What is the epectation when the hint addr is below 128TB but addr + len
>>
>128TB ? Should such mmap request fail ?
>
>
>Yes, I believe so.

This *better* be conditional on some kind of settable limit.  Having a barrier in the middle of the address space for no apparent reason to "clean" software is insane.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-03-20 18:09 UTC|newest]

Thread overview: 98+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-13  5:49 [PATCH 00/26] x86: 5-level paging enabling for v4.12 Kirill A. Shutemov
2017-03-13  5:49 ` Kirill A. Shutemov
2017-03-13  5:49 ` Kirill A. Shutemov
2017-03-13  5:49 ` [PATCH 01/26] x86: basic changes into headers for 5-level paging Kirill A. Shutemov
2017-03-13  5:49   ` Kirill A. Shutemov
2017-03-13  5:49 ` [PATCH 02/26] x86: trivial portion of 5-level paging conversion Kirill A. Shutemov
2017-03-13  5:49   ` Kirill A. Shutemov
2017-03-13  5:49 ` [PATCH 03/26] x86/gup: add 5-level paging support Kirill A. Shutemov
2017-03-13  5:49   ` Kirill A. Shutemov
2017-03-13  5:49 ` [PATCH 04/26] x86/ident_map: " Kirill A. Shutemov
2017-03-13  5:49   ` Kirill A. Shutemov
2017-03-13  5:49 ` [PATCH 05/26] x86/mm: add support of p4d_t in vmalloc_fault() Kirill A. Shutemov
2017-03-13  5:49   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 06/26] x86/power: support p4d_t in hibernate code Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 07/26] x86/kexec: support p4d_t Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 08/26] x86/efi: handle p4d in EFI pagetables Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 09/26] x86/mm/pat: handle additional page table Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 10/26] x86/kasan: prepare clear_pgds() to switch to <asm-generic/pgtable-nop4d.h> Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 11/26] x86/xen: convert __xen_pgd_walk() and xen_cleanmfnmap() to support p4d Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 12/26] x86: convert the rest of the code to support p4d_t Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 13/26] x86: detect 5-level paging support Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 14/26] x86/asm: remove __VIRTUAL_MASK_SHIFT==47 assert Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 15/26] x86/mm: define virtual memory map for 5-level paging Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 16/26] x86/paravirt: make paravirt code support " Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 17/26] x86/mm: basic defines/helpers for CONFIG_X86_5LEVEL Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 18/26] x86/dump_pagetables: support 5-level paging Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 19/26] x86/kasan: extend to " Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  7:25   ` Dmitry Vyukov
2017-03-13  7:25     ` Dmitry Vyukov
2017-03-13  5:50 ` [PATCH 20/26] x86/espfix: " Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 21/26] x86/mm: add support of additional page table level during early boot Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  7:18   ` Ingo Molnar
2017-03-13  7:18     ` Ingo Molnar
2017-04-05 11:36     ` Kirill A. Shutemov
2017-04-05 11:36       ` Kirill A. Shutemov
2017-04-05 15:32       ` Kirill A. Shutemov
2017-04-05 15:32         ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 22/26] x86/mm: add sync_global_pgds() for configuration with 5-level paging Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  7:22   ` Ingo Molnar
2017-03-13  7:22     ` Ingo Molnar
2017-03-13  5:50 ` [PATCH 23/26] x86/mm: make kernel_physical_mapping_init() support " Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 24/26] x86/mm: add support for 5-level paging for KASLR Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 25/26] x86: enable 5-level paging support Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-13  5:50 ` [PATCH 26/26] x86/mm: allow to have userspace mappings above 47-bits Kirill A. Shutemov
2017-03-13  5:50   ` Kirill A. Shutemov
2017-03-17 17:53   ` Aneesh Kumar K.V
2017-03-17 17:53     ` Aneesh Kumar K.V
2017-03-17 17:53     ` Aneesh Kumar K.V
2017-03-17 17:53     ` Aneesh Kumar K.V
2017-03-17 17:57     ` Kirill A. Shutemov
2017-03-17 17:57       ` Kirill A. Shutemov
2017-03-19  8:24       ` Aneesh Kumar K.V
2017-03-19  8:24         ` Aneesh Kumar K.V
2017-03-19  8:26         ` Kirill A. Shutemov
2017-03-20 18:08           ` hpa [this message]
2017-03-20 18:08             ` hpa
2017-03-20 18:38             ` Matthew Wilcox
2017-03-20 18:38               ` Matthew Wilcox
2017-03-24  8:59             ` Kirill A. Shutemov
2017-03-24  8:59               ` Kirill A. Shutemov
2017-03-19  8:55         ` Aneesh Kumar K.V
2017-03-19  8:55           ` Aneesh Kumar K.V
2017-03-24  9:03           ` Kirill A. Shutemov
2017-03-24  9:03             ` Kirill A. Shutemov
2017-03-20  9:15         ` Michael Ellerman
2017-03-20  9:15           ` Michael Ellerman
2017-03-20  5:10   ` Aneesh Kumar K.V
2017-03-20  5:10     ` Aneesh Kumar K.V
2017-03-20  5:10     ` Aneesh Kumar K.V
2017-03-20  5:10     ` Aneesh Kumar K.V
2017-03-24  9:04     ` Kirill A. Shutemov
2017-03-24  9:04       ` Kirill A. Shutemov
2017-03-24  9:14       ` Aneesh Kumar K.V
2017-03-24  9:14         ` Aneesh Kumar K.V
2017-03-24  9:30         ` Kirill A. Shutemov
2017-03-24  9:30           ` Kirill A. Shutemov
2017-03-13  7:49 ` [PATCH 00/26] x86: 5-level paging enabling for v4.12 Ingo Molnar
2017-03-13  7:49   ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=95631D05-2CA2-4967-A29E-DB396C76F62D@zytor.com \
    --to=hpa@zytor.com \
    --cc=ak@linux.intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=aneesh.kumar@linux.vnet.ibm.com \
    --cc=arnd@arndb.de \
    --cc=dave.hansen@intel.com \
    --cc=kirill.shutemov@linux.intel.com \
    --cc=kirill@shutemov.name \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@amacapital.net \
    --cc=mhocko@suse.com \
    --cc=mingo@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.