All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michael Ellerman <mpe@ellerman.id.au>
To: linux-arm-kernel@lists.infradead.org
Subject: Re: linux-next: Tree for Nov 7
Date: Tue, 14 Nov 2017 09:02:09 +0000	[thread overview]
Message-ID: <87efp1w7vy.fsf@concordia.ellerman.id.au> (raw)
In-Reply-To: <20171113151641.yfqrecpcxllpn5mq@dhcp22.suse.cz>

Michal Hocko <mhocko@kernel.org> writes:

> On Mon 13-11-17 13:00:57, Michal Hocko wrote:
> [...]
>> Yes, I have mentioned that in the previous email but the amount of code
>> would be even larger. Basically every arch which reimplements
>> arch_get_unmapped_area would have to special case new MAP_FIXED flag to
>> do vma lookup.
>
> It turned out that this might be much more easier than I thought after
> all. It seems we can really handle that in the common code.

Ah nice. I should have read this before replying to your previous mail.

> This would mean that we are exposing a new functionality to the userspace though.
> Myabe this would be useful on its own though.

Yes I think it would. At least jemalloc seems like it could use it:

  https://github.com/jemalloc/jemalloc/blob/9f455e2786685b443201c33119765c8093461174/src/pages.c#L65

And I have memories of some JIT code I read once which did a loop of
mmap()s or something to try and get allocations below 4GB or some other
limit - but I can't remember now what it was.

> Just a quick draft (not
> even compile tested) whether this makes sense in general. I would be
> worried about unexpected behavior when somebody set other bit without a
> good reason and we might fail with ENOMEM for such a call now.
>
> Elf loader would then use MAP_FIXED_SAFE rather than MAP_FIXED.
> ---
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..d021c21f9b01 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -31,6 +31,9 @@
>  #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
>  
> +#define MAP_KEEP_MAPPING 0x2000000
> +#define MAP_FIXED_SAFE	MAP_FIXED|MAP_KEEP_MAPPING /* enforce MAP_FIXED without clobbering an existing mapping */


So bike-shedding a bit, but I think "SAFE" is too vague a name.

Perhaps MAP_NO_CLOBBER - which has the single semantic of "do not
clobber any existing mappings".

It would be a flag on its own, so you could pass it with or without
MAP_FIXED, but it would only change the behaviour when MAP_FIXED is
specified also.

cheers

WARNING: multiple messages have this Message-ID (diff)
From: Michael Ellerman <mpe@ellerman.id.au>
To: Michal Hocko <mhocko@kernel.org>
Cc: Joel Stanley <joel@jms.id.au>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux-Next Mailing List <linux-next@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Russell King <linux@armlinux.org.uk>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Abdul Haleem <abdhalee@linux.vnet.ibm.com>,
	Ralf Baechle <ralf@linux-mips.org>,
	"James E.J. Bottomley" <jejb@parisc-linux.org>,
	Helge Deller <deller@gmx.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Rich Felker <dalias@libc.org>,
	"David S. Miller" <davem@davemloft.net>,
	Chris Zankel <chris@zankel.net>,
	Max Filippov <jcmvbkbc@gmail.com>,
	linux-arm-kernel@lists.infradead.org,
	linuxppc-dev@lists.ozlabs.org, linux-mips@linux-mips.org,
Subject: Re: linux-next: Tree for Nov 7
Date: Tue, 14 Nov 2017 20:02:09 +1100	[thread overview]
Message-ID: <87efp1w7vy.fsf@concordia.ellerman.id.au> (raw)
In-Reply-To: <20171113151641.yfqrecpcxllpn5mq@dhcp22.suse.cz>

Michal Hocko <mhocko@kernel.org> writes:

> On Mon 13-11-17 13:00:57, Michal Hocko wrote:
> [...]
>> Yes, I have mentioned that in the previous email but the amount of code
>> would be even larger. Basically every arch which reimplements
>> arch_get_unmapped_area would have to special case new MAP_FIXED flag to
>> do vma lookup.
>
> It turned out that this might be much more easier than I thought after
> all. It seems we can really handle that in the common code.

Ah nice. I should have read this before replying to your previous mail.

> This would mean that we are exposing a new functionality to the userspace though.
> Myabe this would be useful on its own though.

Yes I think it would. At least jemalloc seems like it could use it:

  https://github.com/jemalloc/jemalloc/blob/9f455e2786685b443201c33119765c8093461174/src/pages.c#L65

And I have memories of some JIT code I read once which did a loop of
mmap()s or something to try and get allocations below 4GB or some other
limit - but I can't remember now what it was.

> Just a quick draft (not
> even compile tested) whether this makes sense in general. I would be
> worried about unexpected behavior when somebody set other bit without a
> good reason and we might fail with ENOMEM for such a call now.
>
> Elf loader would then use MAP_FIXED_SAFE rather than MAP_FIXED.
> ---
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..d021c21f9b01 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -31,6 +31,9 @@
>  #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
>  
> +#define MAP_KEEP_MAPPING 0x2000000
> +#define MAP_FIXED_SAFE	MAP_FIXED|MAP_KEEP_MAPPING /* enforce MAP_FIXED without clobbering an existing mapping */


So bike-shedding a bit, but I think "SAFE" is too vague a name.

Perhaps MAP_NO_CLOBBER - which has the single semantic of "do not
clobber any existing mappings".

It would be a flag on its own, so you could pass it with or without
MAP_FIXED, but it would only change the behaviour when MAP_FIXED is
specified also.

cheers

WARNING: multiple messages have this Message-ID (diff)
From: Michael Ellerman <mpe@ellerman.id.au>
To: Michal Hocko <mhocko@kernel.org>
Cc: Joel Stanley <joel@jms.id.au>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux-Next Mailing List <linux-next@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Russell King <linux@armlinux.org.uk>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Abdul Haleem <abdhalee@linux.vnet.ibm.com>,
	Ralf Baechle <ralf@linux-mips.org>,
	"James E.J. Bottomley" <jejb@parisc-linux.org>,
	Helge Deller <deller@gmx.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Rich Felker <dalias@libc.org>,
	"David S. Miller" <davem@davemloft.net>,
	Chris Zankel <chris@zankel.net>,
	Max Filippov <jcmvbkbc@gmail.com>,
	linux-arm-kernel@lists.infradead.org,
	linuxppc-dev@lists.ozlabs.org, linux-mips@linux-mips.org,
	linux-parisc@vger.kernel.org, linux-sh@vger.kernel.org,
	sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org
Subject: Re: linux-next: Tree for Nov 7
Date: Tue, 14 Nov 2017 20:02:09 +1100	[thread overview]
Message-ID: <87efp1w7vy.fsf@concordia.ellerman.id.au> (raw)
In-Reply-To: <20171113151641.yfqrecpcxllpn5mq@dhcp22.suse.cz>

Michal Hocko <mhocko@kernel.org> writes:

> On Mon 13-11-17 13:00:57, Michal Hocko wrote:
> [...]
>> Yes, I have mentioned that in the previous email but the amount of code
>> would be even larger. Basically every arch which reimplements
>> arch_get_unmapped_area would have to special case new MAP_FIXED flag to
>> do vma lookup.
>
> It turned out that this might be much more easier than I thought after
> all. It seems we can really handle that in the common code.

Ah nice. I should have read this before replying to your previous mail.

> This would mean that we are exposing a new functionality to the userspace though.
> Myabe this would be useful on its own though.

Yes I think it would. At least jemalloc seems like it could use it:

  https://github.com/jemalloc/jemalloc/blob/9f455e2786685b443201c33119765c8093461174/src/pages.c#L65

And I have memories of some JIT code I read once which did a loop of
mmap()s or something to try and get allocations below 4GB or some other
limit - but I can't remember now what it was.

> Just a quick draft (not
> even compile tested) whether this makes sense in general. I would be
> worried about unexpected behavior when somebody set other bit without a
> good reason and we might fail with ENOMEM for such a call now.
>
> Elf loader would then use MAP_FIXED_SAFE rather than MAP_FIXED.
> ---
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..d021c21f9b01 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -31,6 +31,9 @@
>  #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
>  
> +#define MAP_KEEP_MAPPING 0x2000000
> +#define MAP_FIXED_SAFE	MAP_FIXED|MAP_KEEP_MAPPING /* enforce MAP_FIXED without clobbering an existing mapping */


So bike-shedding a bit, but I think "SAFE" is too vague a name.

Perhaps MAP_NO_CLOBBER - which has the single semantic of "do not
clobber any existing mappings".

It would be a flag on its own, so you could pass it with or without
MAP_FIXED, but it would only change the behaviour when MAP_FIXED is
specified also.

cheers

WARNING: multiple messages have this Message-ID (diff)
From: Michael Ellerman <mpe@ellerman.id.au>
To: Michal Hocko <mhocko@kernel.org>
Cc: Joel Stanley <joel@jms.id.au>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux-Next Mailing List <linux-next@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Russell King <linux@armlinux.org.uk>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Abdul Haleem <abdhalee@linux.vnet.ibm.com>,
	Ralf Baechle <ralf@linux-mips.org>,
	"James E.J. Bottomley" <jejb@parisc-linux.org>,
	Helge Deller <deller@gmx.de>,
	Yoshinori Sato <ysato@users.sourceforge.jp>,
	Rich Felker <dalias@libc.org>,
	"David S. Miller" <davem@davemloft.net>,
	Chris Zankel <chris@zankel.net>,
	Max Filippov <jcmvbkbc@gmail.com>,
	linux-arm-kernel@lists.infradead.org,
	linuxppc-dev@lists.ozlabs.org, linux-mips@linux-mips.org
Subject: Re: linux-next: Tree for Nov 7
Date: Tue, 14 Nov 2017 20:02:09 +1100	[thread overview]
Message-ID: <87efp1w7vy.fsf@concordia.ellerman.id.au> (raw)
In-Reply-To: <20171113151641.yfqrecpcxllpn5mq@dhcp22.suse.cz>

Michal Hocko <mhocko@kernel.org> writes:

> On Mon 13-11-17 13:00:57, Michal Hocko wrote:
> [...]
>> Yes, I have mentioned that in the previous email but the amount of code
>> would be even larger. Basically every arch which reimplements
>> arch_get_unmapped_area would have to special case new MAP_FIXED flag to
>> do vma lookup.
>
> It turned out that this might be much more easier than I thought after
> all. It seems we can really handle that in the common code.

Ah nice. I should have read this before replying to your previous mail.

> This would mean that we are exposing a new functionality to the userspace though.
> Myabe this would be useful on its own though.

Yes I think it would. At least jemalloc seems like it could use it:

  https://github.com/jemalloc/jemalloc/blob/9f455e2786685b443201c33119765c8093461174/src/pages.c#L65

And I have memories of some JIT code I read once which did a loop of
mmap()s or something to try and get allocations below 4GB or some other
limit - but I can't remember now what it was.

> Just a quick draft (not
> even compile tested) whether this makes sense in general. I would be
> worried about unexpected behavior when somebody set other bit without a
> good reason and we might fail with ENOMEM for such a call now.
>
> Elf loader would then use MAP_FIXED_SAFE rather than MAP_FIXED.
> ---
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..d021c21f9b01 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -31,6 +31,9 @@
>  #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
>  
> +#define MAP_KEEP_MAPPING 0x2000000
> +#define MAP_FIXED_SAFE	MAP_FIXED|MAP_KEEP_MAPPING /* enforce MAP_FIXED without clobbering an existing mapping */


So bike-shedding a bit, but I think "SAFE" is too vague a name.

Perhaps MAP_NO_CLOBBER - which has the single semantic of "do not
clobber any existing mappings".

It would be a flag on its own, so you could pass it with or without
MAP_FIXED, but it would only change the behaviour when MAP_FIXED is
specified also.

cheers

WARNING: multiple messages have this Message-ID (diff)
From: mpe@ellerman.id.au (Michael Ellerman)
To: linux-arm-kernel@lists.infradead.org
Subject: linux-next: Tree for Nov 7
Date: Tue, 14 Nov 2017 20:02:09 +1100	[thread overview]
Message-ID: <87efp1w7vy.fsf@concordia.ellerman.id.au> (raw)
In-Reply-To: <20171113151641.yfqrecpcxllpn5mq@dhcp22.suse.cz>

Michal Hocko <mhocko@kernel.org> writes:

> On Mon 13-11-17 13:00:57, Michal Hocko wrote:
> [...]
>> Yes, I have mentioned that in the previous email but the amount of code
>> would be even larger. Basically every arch which reimplements
>> arch_get_unmapped_area would have to special case new MAP_FIXED flag to
>> do vma lookup.
>
> It turned out that this might be much more easier than I thought after
> all. It seems we can really handle that in the common code.

Ah nice. I should have read this before replying to your previous mail.

> This would mean that we are exposing a new functionality to the userspace though.
> Myabe this would be useful on its own though.

Yes I think it would. At least jemalloc seems like it could use it:

  https://github.com/jemalloc/jemalloc/blob/9f455e2786685b443201c33119765c8093461174/src/pages.c#L65

And I have memories of some JIT code I read once which did a loop of
mmap()s or something to try and get allocations below 4GB or some other
limit - but I can't remember now what it was.

> Just a quick draft (not
> even compile tested) whether this makes sense in general. I would be
> worried about unexpected behavior when somebody set other bit without a
> good reason and we might fail with ENOMEM for such a call now.
>
> Elf loader would then use MAP_FIXED_SAFE rather than MAP_FIXED.
> ---
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..d021c21f9b01 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -31,6 +31,9 @@
>  #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
>  #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
>  
> +#define MAP_KEEP_MAPPING 0x2000000
> +#define MAP_FIXED_SAFE	MAP_FIXED|MAP_KEEP_MAPPING /* enforce MAP_FIXED without clobbering an existing mapping */


So bike-shedding a bit, but I think "SAFE" is too vague a name.

Perhaps MAP_NO_CLOBBER - which has the single semantic of "do not
clobber any existing mappings".

It would be a flag on its own, so you could pass it with or without
MAP_FIXED, but it would only change the behaviour when MAP_FIXED is
specified also.

cheers

  parent reply	other threads:[~2017-11-14  9:02 UTC|newest]

Thread overview: 102+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-07  5:22 linux-next: Tree for Nov 7 Stephen Rothwell
2017-11-07 22:22 ` Joel Stanley
2017-11-08 14:20   ` Michal Hocko
2017-11-10 12:30     ` Michal Hocko
2017-11-12  1:08       ` Joel Stanley
2017-11-13  9:20         ` Michal Hocko
2017-11-13  9:20           ` Michal Hocko
2017-11-13  9:34           ` Russell King - ARM Linux
2017-11-13  9:34             ` Russell King - ARM Linux
2017-11-13  9:42           ` Michal Hocko
2017-11-13  9:42             ` Michal Hocko
2017-11-13  9:42             ` Michal Hocko
2017-11-13  9:42             ` Michal Hocko
2017-11-13 11:34             ` Michael Ellerman
2017-11-13 11:34               ` Michael Ellerman
2017-11-13 11:34               ` Michael Ellerman
2017-11-13 11:34               ` Michael Ellerman
2017-11-13 11:34               ` Michael Ellerman
2017-11-13 12:00               ` Michal Hocko
2017-11-13 12:00                 ` Michal Hocko
2017-11-13 12:00                 ` Michal Hocko
2017-11-13 12:00                 ` Michal Hocko
2017-11-13 12:00                 ` Michal Hocko
2017-11-13 15:16                 ` Michal Hocko
2017-11-13 15:16                   ` Michal Hocko
2017-11-13 15:16                   ` Michal Hocko
2017-11-13 15:16                   ` Michal Hocko
2017-11-13 15:16                   ` Michal Hocko
2017-11-13 15:48                   ` Russell King - ARM Linux
2017-11-13 15:48                     ` Russell King - ARM Linux
2017-11-13 15:48                     ` Russell King - ARM Linux
2017-11-13 15:48                     ` Russell King - ARM Linux
2017-11-13 15:48                     ` Russell King - ARM Linux
2017-11-13 15:59                     ` Michal Hocko
2017-11-13 15:59                       ` Michal Hocko
2017-11-13 15:59                       ` Michal Hocko
2017-11-13 15:59                       ` Michal Hocko
2017-11-13 15:59                       ` Michal Hocko
2017-11-13 15:49                   ` Michal Hocko
2017-11-13 15:49                     ` Michal Hocko
2017-11-13 15:49                     ` Michal Hocko
2017-11-13 15:49                     ` Michal Hocko
2017-11-13 15:49                     ` Michal Hocko
2017-11-13 16:06                     ` Michal Hocko
2017-11-13 16:06                       ` Michal Hocko
2017-11-13 16:06                       ` Michal Hocko
2017-11-13 16:06                       ` Michal Hocko
2017-11-13 16:06                       ` Michal Hocko
2017-11-13 16:35                       ` Khalid Aziz
2017-11-13 16:35                         ` Khalid Aziz
2017-11-13 16:35                         ` Khalid Aziz
2017-11-13 16:35                         ` Khalid Aziz
2017-11-13 16:35                         ` Khalid Aziz
2017-11-14  7:07                         ` Michal Hocko
2017-11-14  7:07                           ` Michal Hocko
2017-11-14  7:07                           ` Michal Hocko
2017-11-14  7:07                           ` Michal Hocko
2017-11-14  9:18                       ` Michael Ellerman
2017-11-14  9:18                         ` Michael Ellerman
2017-11-14  9:18                         ` Michael Ellerman
2017-11-14  9:18                         ` Michael Ellerman
2017-11-14  9:18                         ` Michael Ellerman
2017-11-14  9:29                         ` Michal Hocko
2017-11-14  9:29                           ` Michal Hocko
2017-11-14  9:29                           ` Michal Hocko
2017-11-14  9:29                           ` Michal Hocko
2017-11-14  9:29                           ` Michal Hocko
2017-11-14  9:02                   ` Michael Ellerman [this message]
2017-11-14  9:02                     ` Michael Ellerman
2017-11-14  9:02                     ` Michael Ellerman
2017-11-14  9:02                     ` Michael Ellerman
2017-11-14  9:02                     ` Michael Ellerman
2017-11-14  8:54                 ` Michael Ellerman
2017-11-14  8:54                   ` Michael Ellerman
2017-11-14  8:54                   ` Michael Ellerman
2017-11-14  8:54                   ` Michael Ellerman
2017-11-14  8:54                   ` Michael Ellerman
2017-11-14  9:04                   ` Michal Hocko
2017-11-14  9:04                     ` Michal Hocko
2017-11-14  9:04                     ` Michal Hocko
2017-11-14  9:04                     ` Michal Hocko
2017-11-14  9:04                     ` Michal Hocko
2017-11-14 14:52                     ` Khalid Aziz
2017-11-14 14:52                       ` Khalid Aziz
2017-11-14 14:52                       ` Khalid Aziz
2017-11-14 14:52                       ` Khalid Aziz
2017-11-13 14:11           ` Michal Hocko
2017-11-13 14:11             ` Michal Hocko
2017-11-13 15:09             ` Russell King - ARM Linux
2017-11-13 15:09               ` Russell King - ARM Linux
2017-11-13 15:31               ` Michal Hocko
2017-11-13 15:31                 ` Michal Hocko
2017-11-14  0:03         ` Andrew Morton
2017-11-14  0:36           ` Stephen Rothwell
  -- strict thread matches above, loose matches on Subject: below --
2023-11-07  1:17 Stephen Rothwell
2022-11-07  3:44 Stephen Rothwell
2019-11-07  8:07 Stephen Rothwell
2018-11-07  3:16 Stephen Rothwell
2013-11-07  7:31 Stephen Rothwell
2013-11-07  7:31 ` Stephen Rothwell
2012-11-07  5:15 Stephen Rothwell
2012-11-07  5:15 ` Stephen Rothwell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87efp1w7vy.fsf@concordia.ellerman.id.au \
    --to=mpe@ellerman.id.au \
    --cc=linux-arm-kernel@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.