* remap_file_pages regression
@ 2016-02-15 1:32 Grazvydas Ignotas
2016-02-15 10:26 ` Kirill A. Shutemov
0 siblings, 1 reply; 4+ messages in thread
From: Grazvydas Ignotas @ 2016-02-15 1:32 UTC (permalink / raw)
To: linux-kernel, Kirill A . Shutemov; +Cc: Andrew Morton, Grazvydas Ignotas
Hi,
since remap_file_pages() rework the following simple program fails.
I haven't actually bisected this, only know it worked on 3.19 at least
(I bought a new system now and need 4.2+ for hardware support). If you
are curious, the program is an emulator and is using remap_file_pages()
to implement memory mirroring efficiently (and to remap things during
run time).
Grazvydas
====%<===
#define _GNU_SOURCE
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>
#define SIZE (4096 * 3)
int main(int argc, char **argv)
{
unsigned long *p;
long i;
p = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (p == MAP_FAILED) {
perror("mmap");
return -1;
}
for (i = 0; i < SIZE / 4096; i++)
p[i * 4096 / sizeof(*p)] = i;
if (remap_file_pages(p, 4096, 0, 1, 0)) {
perror("remap_file_pages");
return -1;
}
if (remap_file_pages(p, 4096 * 2, 0, 1, 0)) {
perror("remap_file_pages");
return -1;
}
assert(p[0] == 1);
munmap(p, SIZE);
return 0;
}
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: remap_file_pages regression
2016-02-15 1:32 remap_file_pages regression Grazvydas Ignotas
@ 2016-02-15 10:26 ` Kirill A. Shutemov
2016-02-15 22:08 ` Grazvydas Ignotas
0 siblings, 1 reply; 4+ messages in thread
From: Kirill A. Shutemov @ 2016-02-15 10:26 UTC (permalink / raw)
To: Grazvydas Ignotas; +Cc: linux-kernel, Andrew Morton
On Mon, Feb 15, 2016 at 03:32:55AM +0200, Grazvydas Ignotas wrote:
> Hi,
>
> since remap_file_pages() rework the following simple program fails.
> I haven't actually bisected this, only know it worked on 3.19 at least
> (I bought a new system now and need 4.2+ for hardware support).
The patch below should fix the issue. Please test.
> If you are curious, the program is an emulator and is using
> remap_file_pages() to implement memory mirroring efficiently (and to
> remap things during run time).
Could you elaborate on this?
Why creating file on tmpfs/shmem (using memfd_create() for example) plus
plain mmap()s wouldn't work for you?
>From 6b8690c3a983a72212db8cd35eb4cab106a195d4 Mon Sep 17 00:00:00 2001
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Date: Mon, 15 Feb 2016 13:06:16 +0300
Subject: [PATCH] mm: fix regression in remap_file_pages() emulation
Grazvydas Ignotas has reported a regression in remap_file_pages()
emulation.
Testcase:
#define _GNU_SOURCE
#include <assert.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/mman.h>
#define SIZE (4096 * 3)
int main(int argc, char **argv)
{
unsigned long *p;
long i;
p = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (p == MAP_FAILED) {
perror("mmap");
return -1;
}
for (i = 0; i < SIZE / 4096; i++)
p[i * 4096 / sizeof(*p)] = i;
if (remap_file_pages(p, 4096, 0, 1, 0)) {
perror("remap_file_pages");
return -1;
}
if (remap_file_pages(p, 4096 * 2, 0, 1, 0)) {
perror("remap_file_pages");
return -1;
}
assert(p[0] == 1);
munmap(p, SIZE);
return 0;
}
The second remap_file_pages() fails with -EINVAL.
The reason is that remap_file_pages() emulation assumes that the target
vma covers whole area we want to over map. That assumption is broken by
first remap_file_pages() call: it split the area into two vma.
The solution is to check next adjacent vmas, if they map the same file
with the same flags.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reported-by: Grazvydas Ignotas <notasas@gmail.com>
---
mm/mmap.c | 34 +++++++++++++++++++++++++++++-----
1 file changed, 29 insertions(+), 5 deletions(-)
diff --git a/mm/mmap.c b/mm/mmap.c
index 2f2415a7a688..76d1ec29149b 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -2664,12 +2664,29 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
if (!vma || !(vma->vm_flags & VM_SHARED))
goto out;
- if (start < vma->vm_start || start + size > vma->vm_end)
+ if (start < vma->vm_start)
goto out;
- if (pgoff == linear_page_index(vma, start)) {
- ret = 0;
- goto out;
+ if (start + size > vma->vm_end) {
+ struct vm_area_struct *next;
+
+ for (next = vma->vm_next; next; next = next->vm_next) {
+ /* hole between vmas ? */
+ if (next->vm_start != next->vm_prev->vm_end)
+ goto out;
+
+ if (next->vm_file != vma->vm_file)
+ goto out;
+
+ if (next->vm_flags != vma->vm_flags)
+ goto out;
+
+ if (start + size <= next->vm_end)
+ break;
+ }
+
+ if (!next)
+ goto out;
}
prot |= vma->vm_flags & VM_READ ? PROT_READ : 0;
@@ -2679,9 +2696,16 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
flags &= MAP_NONBLOCK;
flags |= MAP_SHARED | MAP_FIXED | MAP_POPULATE;
if (vma->vm_flags & VM_LOCKED) {
+ struct vm_area_struct *tmp;
flags |= MAP_LOCKED;
+
/* drop PG_Mlocked flag for over-mapped range */
- munlock_vma_pages_range(vma, start, start + size);
+ for (tmp = vma; tmp->vm_start >= start + size;
+ tmp = tmp->vm_next) {
+ munlock_vma_pages_range(tmp,
+ max(tmp->vm_start, start),
+ min(tmp->vm_end, start + size));
+ }
}
file = get_file(vma->vm_file);
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: remap_file_pages regression
2016-02-15 10:26 ` Kirill A. Shutemov
@ 2016-02-15 22:08 ` Grazvydas Ignotas
2016-02-15 22:17 ` Kirill A. Shutemov
0 siblings, 1 reply; 4+ messages in thread
From: Grazvydas Ignotas @ 2016-02-15 22:08 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: linux-kernel, Andrew Morton
On Mon, Feb 15, 2016 at 12:26 PM, Kirill A. Shutemov
<kirill.shutemov@linux.intel.com> wrote:
> On Mon, Feb 15, 2016 at 03:32:55AM +0200, Grazvydas Ignotas wrote:
>> Hi,
>>
>> since remap_file_pages() rework the following simple program fails.
>> I haven't actually bisected this, only know it worked on 3.19 at least
>> (I bought a new system now and need 4.2+ for hardware support).
>
> The patch below should fix the issue. Please test.
It works, thanks.
Tested-by: Grazvydas Ignotas <notasas@gmail.com>
>
>> If you are curious, the program is an emulator and is using
>> remap_file_pages() to implement memory mirroring efficiently (and to
>> remap things during run time).
>
> Could you elaborate on this?
>
> Why creating file on tmpfs/shmem (using memfd_create() for example) plus
> plain mmap()s wouldn't work for you?
It works, but remap_file_pages() is just more convenient, you don't
need many mmap()/munmap() calls (less syscalls), and as the emulator
needs to reconfigure the mappings during runtime (it's not a one time
init thing), remap_file_pages() makes more sense. The reduced number
of VMAs of the past was also a benefit I guess.
Actually I'm not the author of the emulator in question, so I've asked
the author to comment:
--- quote ---
One of the things we do with remap_file_pages is a mapping for the
emulated system's VRAM. The system allows many different
configurations of several differently sized VRAM banks to a 16MB area
of address space. The banks are multiples of 16KB in size and their
allocations are also 16KB aligned. With remap_file_pages we can map
that entire space to an (in-memory) file and then arbitrarily map
chunks of it as desired. We have an mmap-based fallback, but it
requires us to manage a separate mmap for each 16KB region."
--- end of quote ---
Apparently the mentioned fallback was a compile time option, so I was
hit with the issue after upgrading my machine.
> <snip>
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: stable?
Fixes: c8d78c182 (I guess?)
> Reported-by: Grazvydas Ignotas <notasas@gmail.com>
> ---
> mm/mmap.c | 34 +++++++++++++++++++++++++++++-----
> 1 file changed, 29 insertions(+), 5 deletions(-)
>
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 2f2415a7a688..76d1ec29149b 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -2664,12 +2664,29 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
> if (!vma || !(vma->vm_flags & VM_SHARED))
> goto out;
>
> - if (start < vma->vm_start || start + size > vma->vm_end)
> + if (start < vma->vm_start)
> goto out;
>
> - if (pgoff == linear_page_index(vma, start)) {
> - ret = 0;
> - goto out;
> + if (start + size > vma->vm_end) {
> + struct vm_area_struct *next;
> +
> + for (next = vma->vm_next; next; next = next->vm_next) {
> + /* hole between vmas ? */
> + if (next->vm_start != next->vm_prev->vm_end)
> + goto out;
> +
> + if (next->vm_file != vma->vm_file)
> + goto out;
> +
> + if (next->vm_flags != vma->vm_flags)
> + goto out;
> +
> + if (start + size <= next->vm_end)
> + break;
> + }
> +
> + if (!next)
> + goto out;
> }
>
> prot |= vma->vm_flags & VM_READ ? PROT_READ : 0;
> @@ -2679,9 +2696,16 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
> flags &= MAP_NONBLOCK;
> flags |= MAP_SHARED | MAP_FIXED | MAP_POPULATE;
> if (vma->vm_flags & VM_LOCKED) {
> + struct vm_area_struct *tmp;
> flags |= MAP_LOCKED;
> +
> /* drop PG_Mlocked flag for over-mapped range */
> - munlock_vma_pages_range(vma, start, start + size);
> + for (tmp = vma; tmp->vm_start >= start + size;
> + tmp = tmp->vm_next) {
> + munlock_vma_pages_range(tmp,
> + max(tmp->vm_start, start),
> + min(tmp->vm_end, start + size));
> + }
> }
>
> file = get_file(vma->vm_file);
> --
> Kirill A. Shutemov
Gražvydas
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: remap_file_pages regression
2016-02-15 22:08 ` Grazvydas Ignotas
@ 2016-02-15 22:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 4+ messages in thread
From: Kirill A. Shutemov @ 2016-02-15 22:17 UTC (permalink / raw)
To: Grazvydas Ignotas; +Cc: linux-kernel, Andrew Morton
On Tue, Feb 16, 2016 at 12:08:34AM +0200, Grazvydas Ignotas wrote:
> On Mon, Feb 15, 2016 at 12:26 PM, Kirill A. Shutemov
> <kirill.shutemov@linux.intel.com> wrote:
> > On Mon, Feb 15, 2016 at 03:32:55AM +0200, Grazvydas Ignotas wrote:
> >> Hi,
> >>
> >> since remap_file_pages() rework the following simple program fails.
> >> I haven't actually bisected this, only know it worked on 3.19 at least
> >> (I bought a new system now and need 4.2+ for hardware support).
> >
> > The patch below should fix the issue. Please test.
>
> It works, thanks.
> Tested-by: Grazvydas Ignotas <notasas@gmail.com>
>
> >
> >> If you are curious, the program is an emulator and is using
> >> remap_file_pages() to implement memory mirroring efficiently (and to
> >> remap things during run time).
> >
> > Could you elaborate on this?
> >
> > Why creating file on tmpfs/shmem (using memfd_create() for example) plus
> > plain mmap()s wouldn't work for you?
>
> It works, but remap_file_pages() is just more convenient, you don't
> need many mmap()/munmap() calls (less syscalls)
mmap(MAP_FIXED) would do munmap() for you.
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Cc: stable?
> Fixes: c8d78c182 (I guess?)
Right.
Cc: stable@vger.kernel.org # v4.0+
Fixes: c8d78c1823f4 ("mm: replace remap_file_pages() syscall with emulation")
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-02-15 22:17 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-15 1:32 remap_file_pages regression Grazvydas Ignotas
2016-02-15 10:26 ` Kirill A. Shutemov
2016-02-15 22:08 ` Grazvydas Ignotas
2016-02-15 22:17 ` Kirill A. Shutemov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.