* [PATCH] perf/core: fix mlock accounting in perf_mmap()
@ 2020-01-17 23:45 Song Liu
2020-01-20 8:24 ` Alexander Shishkin
0 siblings, 1 reply; 7+ messages in thread
From: Song Liu @ 2020-01-17 23:45 UTC (permalink / raw)
To: linux-kernel
Cc: kernel-team, Song Liu, Alexander Shishkin,
Arnaldo Carvalho de Melo, Jiri Olsa, Peter Zijlstra
sysctl_perf_event_mlock and user->locked_vm can change value
independently, so we can't guarantee:
user->locked_vm <= user_lock_limit
When user->locked_vm is larger than user_lock_limit, we cannot simply
update extra and user_extra as:
extra = user_locked - user_lock_limit;
user_extra -= extra;
Otherwise, user_extra will be negative. In extreme cases, this may lead to
negative user->locked_vm (until this perf-mmap is closed), which break
locked_vm badly.
Fix this with two separate conditions, which make sure user_extra is
always positive.
Fixes: c4b75479741c ("perf/core: Make the mlock accounting simple again")
Signed-off-by: Song Liu <songliubraving@fb.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
---
kernel/events/core.c | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a1f8bde19b56..89acdd1574ef 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -5920,11 +5920,31 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
if (user_locked > user_lock_limit) {
/*
- * charge locked_vm until it hits user_lock_limit;
- * charge the rest from pinned_vm
+ * sysctl_perf_event_mlock and user->locked_vm can change
+ * value independently, so we can't guarantee:
+ *
+ * user->locked_vm <= user_lock_limit
+ *
+ * We need be careful to make sure user_extra >=0.
+ *
+ * Using "user_locked - user_extra" to avoid calling
+ * atomic_long_read() again.
*/
- extra = user_locked - user_lock_limit;
- user_extra -= extra;
+ if (user_locked - user_extra >= user_lock_limit) {
+ /*
+ * already used all user_locked_limit, charge all
+ * to pinned_vm
+ */
+ extra = user_extra;
+ user_extra = 0;
+ } else {
+ /*
+ * charge locked_vm until it hits user_lock_limit;
+ * charge the rest from pinned_vm
+ */
+ extra = user_locked - user_lock_limit;
+ user_extra -= extra;
+ }
}
lock_limit = rlimit(RLIMIT_MEMLOCK);
--
2.17.1
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-17 23:45 [PATCH] perf/core: fix mlock accounting in perf_mmap() Song Liu
@ 2020-01-20 8:24 ` Alexander Shishkin
2020-01-21 18:55 ` Song Liu
2020-01-21 19:35 ` Song Liu
0 siblings, 2 replies; 7+ messages in thread
From: Alexander Shishkin @ 2020-01-20 8:24 UTC (permalink / raw)
To: Song Liu, linux-kernel
Cc: kernel-team, Song Liu, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra, alexander.shishkin
Song Liu <songliubraving@fb.com> writes:
> sysctl_perf_event_mlock and user->locked_vm can change value
> independently, so we can't guarantee:
>
> user->locked_vm <= user_lock_limit
This means: if the sysctl got sufficiently decreased, so that the
existing locked_vm exceeds it, we need to deal with the overflow, right?
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index a1f8bde19b56..89acdd1574ef 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -5920,11 +5920,31 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
>
> if (user_locked > user_lock_limit) {
> /*
> - * charge locked_vm until it hits user_lock_limit;
> - * charge the rest from pinned_vm
> + * sysctl_perf_event_mlock and user->locked_vm can change
> + * value independently, so we can't guarantee:
> + *
> + * user->locked_vm <= user_lock_limit
> + *
> + * We need be careful to make sure user_extra >=0.
> + *
> + * Using "user_locked - user_extra" to avoid calling
> + * atomic_long_read() again.
> */
> - extra = user_locked - user_lock_limit;
> - user_extra -= extra;
> + if (user_locked - user_extra >= user_lock_limit) {
> + /*
> + * already used all user_locked_limit, charge all
> + * to pinned_vm
> + */
> + extra = user_extra;
> + user_extra = 0;
> + } else {
> + /*
> + * charge locked_vm until it hits user_lock_limit;
> + * charge the rest from pinned_vm
> + */
> + extra = user_locked - user_lock_limit;
> + user_extra -= extra;
> + }
How about the below for the sake of brevity?
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 763cf34b5a63..632505ce6c12 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -5917,7 +5917,14 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
*/
user_lock_limit *= num_online_cpus();
- user_locked = atomic_long_read(&user->locked_vm) + user_extra;
+ user_locked = atomic_long_read(&user->locked_vm);
+ /*
+ * If perf_event_mlock has changed since earlier mmaps, so that
+ * it's smaller than user->locked_vm, discard the overflow.
+ */
+ if (user_locked > user_lock_limit)
+ user_locked = user_lock_limit;
+ user_locked += user_extra;
if (user_locked > user_lock_limit) {
/*
Regards,
--
Alex
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-20 8:24 ` Alexander Shishkin
@ 2020-01-21 18:55 ` Song Liu
2020-01-23 9:19 ` Alexander Shishkin
2020-01-21 19:35 ` Song Liu
1 sibling, 1 reply; 7+ messages in thread
From: Song Liu @ 2020-01-21 18:55 UTC (permalink / raw)
To: Alexander Shishkin
Cc: linux-kernel, Kernel Team, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra
> On Jan 20, 2020, at 12:24 AM, Alexander Shishkin <alexander.shishkin@linux.intel.com> wrote:
>
> Song Liu <songliubraving@fb.com> writes:
>
>> sysctl_perf_event_mlock and user->locked_vm can change value
>> independently, so we can't guarantee:
>>
>> user->locked_vm <= user_lock_limit
>
> This means: if the sysctl got sufficiently decreased, so that the
> existing locked_vm exceeds it, we need to deal with the overflow, right?
Reducing sysctl is one way to generate the overflow. Another way is to
call setrlimit() from user space to allow bigger user->locked_vm.
>
>> diff --git a/kernel/events/core.c b/kernel/events/core.c
>> index a1f8bde19b56..89acdd1574ef 100644
>> --- a/kernel/events/core.c
>> +++ b/kernel/events/core.c
>> @@ -5920,11 +5920,31 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
>>
>> if (user_locked > user_lock_limit) {
>> /*
>> - * charge locked_vm until it hits user_lock_limit;
>> - * charge the rest from pinned_vm
>> + * sysctl_perf_event_mlock and user->locked_vm can change
>> + * value independently, so we can't guarantee:
>> + *
>> + * user->locked_vm <= user_lock_limit
>> + *
>> + * We need be careful to make sure user_extra >=0.
>> + *
>> + * Using "user_locked - user_extra" to avoid calling
>> + * atomic_long_read() again.
>> */
>> - extra = user_locked - user_lock_limit;
>> - user_extra -= extra;
>> + if (user_locked - user_extra >= user_lock_limit) {
>> + /*
>> + * already used all user_locked_limit, charge all
>> + * to pinned_vm
>> + */
>> + extra = user_extra;
>> + user_extra = 0;
>> + } else {
>> + /*
>> + * charge locked_vm until it hits user_lock_limit;
>> + * charge the rest from pinned_vm
>> + */
>> + extra = user_locked - user_lock_limit;
>> + user_extra -= extra;
>> + }
>
> How about the below for the sake of brevity?
>
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 763cf34b5a63..632505ce6c12 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -5917,7 +5917,14 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
> */
> user_lock_limit *= num_online_cpus();
>
> - user_locked = atomic_long_read(&user->locked_vm) + user_extra;
> + user_locked = atomic_long_read(&user->locked_vm);
> + /*
> + * If perf_event_mlock has changed since earlier mmaps, so that
> + * it's smaller than user->locked_vm, discard the overflow.
> + */
Since changes in perf_event_mlock is not the only reason for the overflow,
we need to revise this comment.
> + if (user_locked > user_lock_limit)
> + user_locked = user_lock_limit;
> + user_locked += user_extra;
>
> if (user_locked > user_lock_limit) {
> /*
I think this is logically correct, and probably easier to follow. Let me
respin v2 based on this version.
Thanks,
Song
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-21 18:55 ` Song Liu
@ 2020-01-23 9:19 ` Alexander Shishkin
2020-01-23 17:24 ` Song Liu
0 siblings, 1 reply; 7+ messages in thread
From: Alexander Shishkin @ 2020-01-23 9:19 UTC (permalink / raw)
To: Song Liu
Cc: linux-kernel, Kernel Team, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra, alexander.shishkin
Song Liu <songliubraving@fb.com> writes:
>> On Jan 20, 2020, at 12:24 AM, Alexander Shishkin <alexander.shishkin@linux.intel.com> wrote:
>>
>> Song Liu <songliubraving@fb.com> writes:
>>
>>> sysctl_perf_event_mlock and user->locked_vm can change value
>>> independently, so we can't guarantee:
>>>
>>> user->locked_vm <= user_lock_limit
>>
>> This means: if the sysctl got sufficiently decreased, so that the
>> existing locked_vm exceeds it, we need to deal with the overflow, right?
>
> Reducing sysctl is one way to generate the overflow. Another way is to
> call setrlimit() from user space to allow bigger user->locked_vm.
You mean RLIMIT_MEMLOCK? That's a limit on mm->pinned_vm. Doesn't affect
user->locked_vm.
Regards,
--
Alex
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-23 9:19 ` Alexander Shishkin
@ 2020-01-23 17:24 ` Song Liu
0 siblings, 0 replies; 7+ messages in thread
From: Song Liu @ 2020-01-23 17:24 UTC (permalink / raw)
To: Alexander Shishkin
Cc: linux-kernel, Kernel Team, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra
> On Jan 23, 2020, at 1:19 AM, Alexander Shishkin <alexander.shishkin@linux.intel.com> wrote:
>
> Song Liu <songliubraving@fb.com> writes:
>
>>> On Jan 20, 2020, at 12:24 AM, Alexander Shishkin <alexander.shishkin@linux.intel.com> wrote:
>>>
>>> Song Liu <songliubraving@fb.com> writes:
>>>
>>>> sysctl_perf_event_mlock and user->locked_vm can change value
>>>> independently, so we can't guarantee:
>>>>
>>>> user->locked_vm <= user_lock_limit
>>>
>>> This means: if the sysctl got sufficiently decreased, so that the
>>> existing locked_vm exceeds it, we need to deal with the overflow, right?
>>
>> Reducing sysctl is one way to generate the overflow. Another way is to
>> call setrlimit() from user space to allow bigger user->locked_vm.
>
> You mean RLIMIT_MEMLOCK? That's a limit on mm->pinned_vm. Doesn't affect
> user->locked_vm.
This depends. For example, bpf_charge_memlock() uses RLIMIT_MEMLOCK as the
limit for user->locked_vm. This makes sense, because the bpf map created by
a process may stay longer than the process.
Thanks,
Song
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-20 8:24 ` Alexander Shishkin
2020-01-21 18:55 ` Song Liu
@ 2020-01-21 19:35 ` Song Liu
2020-01-22 8:50 ` Alexander Shishkin
1 sibling, 1 reply; 7+ messages in thread
From: Song Liu @ 2020-01-21 19:35 UTC (permalink / raw)
To: Alexander Shishkin
Cc: linux-kernel, Kernel Team, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra
> On Jan 20, 2020, at 12:24 AM, Alexander Shishkin <alexander.shishkin@linux.intel.com> wrote:
>
> Song Liu <songliubraving@fb.com> writes:
>
>> sysctl_perf_event_mlock and user->locked_vm can change value
>> independently, so we can't guarantee:
>>
>> user->locked_vm <= user_lock_limit
>
> This means: if the sysctl got sufficiently decreased, so that the
> existing locked_vm exceeds it, we need to deal with the overflow, right?
>
>> diff --git a/kernel/events/core.c b/kernel/events/core.c
>> index a1f8bde19b56..89acdd1574ef 100644
>> --- a/kernel/events/core.c
>> +++ b/kernel/events/core.c
>> @@ -5920,11 +5920,31 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
>>
>> if (user_locked > user_lock_limit) {
>> /*
>> - * charge locked_vm until it hits user_lock_limit;
>> - * charge the rest from pinned_vm
>> + * sysctl_perf_event_mlock and user->locked_vm can change
>> + * value independently, so we can't guarantee:
>> + *
>> + * user->locked_vm <= user_lock_limit
>> + *
>> + * We need be careful to make sure user_extra >=0.
>> + *
>> + * Using "user_locked - user_extra" to avoid calling
>> + * atomic_long_read() again.
>> */
>> - extra = user_locked - user_lock_limit;
>> - user_extra -= extra;
>> + if (user_locked - user_extra >= user_lock_limit) {
>> + /*
>> + * already used all user_locked_limit, charge all
>> + * to pinned_vm
>> + */
>> + extra = user_extra;
>> + user_extra = 0;
>> + } else {
>> + /*
>> + * charge locked_vm until it hits user_lock_limit;
>> + * charge the rest from pinned_vm
>> + */
>> + extra = user_locked - user_lock_limit;
>> + user_extra -= extra;
>> + }
>
> How about the below for the sake of brevity?
>
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 763cf34b5a63..632505ce6c12 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -5917,7 +5917,14 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
> */
> user_lock_limit *= num_online_cpus();
>
> - user_locked = atomic_long_read(&user->locked_vm) + user_extra;
> + user_locked = atomic_long_read(&user->locked_vm);
> + /*
> + * If perf_event_mlock has changed since earlier mmaps, so that
> + * it's smaller than user->locked_vm, discard the overflow.
> + */
> + if (user_locked > user_lock_limit)
> + user_locked = user_lock_limit;
> + user_locked += user_extra;
>
> if (user_locked > user_lock_limit) {
> /*
Actually, I think this is cleaner.
diff --git i/kernel/events/core.c w/kernel/events/core.c
index 2173c23c25b4..debd84fcf9cc 100644
--- i/kernel/events/core.c
+++ w/kernel/events/core.c
@@ -5916,14 +5916,18 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
*/
user_lock_limit *= num_online_cpus();
- user_locked = atomic_long_read(&user->locked_vm) + user_extra;
+ user_locked = atomic_long_read(&user->locked_vm);
if (user_locked > user_lock_limit) {
+ /* charge all to pinned_vm */
+ extra = user_extra;
+ user_extra = 0;
+ } else if (user_lock + user_extra > user_lock_limit)
/*
* charge locked_vm until it hits user_lock_limit;
* charge the rest from pinned_vm
*/
- extra = user_locked - user_lock_limit;
+ extra = user_locked + user_extra - user_lock_limit;
user_extra -= extra;
}
Alexander, does this look good to you?
Thanks,
Song
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH] perf/core: fix mlock accounting in perf_mmap()
2020-01-21 19:35 ` Song Liu
@ 2020-01-22 8:50 ` Alexander Shishkin
0 siblings, 0 replies; 7+ messages in thread
From: Alexander Shishkin @ 2020-01-22 8:50 UTC (permalink / raw)
To: Song Liu
Cc: linux-kernel, Kernel Team, Arnaldo Carvalho de Melo, Jiri Olsa,
Peter Zijlstra, alexander.shishkin
Song Liu <songliubraving@fb.com> writes:
> Actually, I think this is cleaner.
I don't think multiple conditional blocks are cleaner, at least in this
case.
> diff --git i/kernel/events/core.c w/kernel/events/core.c
> index 2173c23c25b4..debd84fcf9cc 100644
> --- i/kernel/events/core.c
> +++ w/kernel/events/core.c
> @@ -5916,14 +5916,18 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)
> */
> user_lock_limit *= num_online_cpus();
>
> - user_locked = atomic_long_read(&user->locked_vm) + user_extra;
> + user_locked = atomic_long_read(&user->locked_vm);
>
> if (user_locked > user_lock_limit) {
> + /* charge all to pinned_vm */
> + extra = user_extra;
> + user_extra = 0;
> + } else if (user_lock + user_extra > user_lock_limit)
You probably mean "user_locked" here.
> /*
> * charge locked_vm until it hits user_lock_limit;
> * charge the rest from pinned_vm
> */
> - extra = user_locked - user_lock_limit;
> + extra = user_locked + user_extra - user_lock_limit;
To me, this is a bit harder to read.
> user_extra -= extra;
> }
>
> Alexander, does this look good to you?
I like to think of this as: we charge the pages to locked_vm until we
exhaust user_lock_limit, and the rest we charge to pinned_vm. Everything
else are just corner cases, and they fit into the same general case. When
we start calculating each corner case in its own block, we just multiply
the potential errors. And there have been errors in this particular path
before. So, the shorter, and the fewer the "if...else if..." statements,
the better it looks to me. Otherwise, it's a matter of preference.
Thanks,
--
Alex
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-01-23 17:24 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-17 23:45 [PATCH] perf/core: fix mlock accounting in perf_mmap() Song Liu
2020-01-20 8:24 ` Alexander Shishkin
2020-01-21 18:55 ` Song Liu
2020-01-23 9:19 ` Alexander Shishkin
2020-01-23 17:24 ` Song Liu
2020-01-21 19:35 ` Song Liu
2020-01-22 8:50 ` Alexander Shishkin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).