BPF Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
@ 2019-11-29 22:29 Daniel Borkmann
  2019-11-30  1:37 ` Eric Dumazet
  0 siblings, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2019-11-29 22:29 UTC (permalink / raw)
  To: alexei.starovoitov; +Cc: peterz, netdev, bpf, Daniel Borkmann

For the case where the interpreter is compiled out or when the prog is jited
it is completely unnecessary to set the BPF insn pages as read-only. In fact,
on frequent churn of BPF programs, it could lead to performance degradation of
the system over time since it would break the direct map down to 4k pages when
calling set_memory_ro() for the insn buffer on x86-64 / arm64 and there is no
reverse operation. Thus, avoid breaking up large pages for data maps, and only
limit this to the module range used by the JIT where it is necessary to set
the image read-only and executable.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 include/linux/filter.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/include/linux/filter.h b/include/linux/filter.h
index 1b1e8b8f88da..a141cb07e76a 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -776,8 +776,12 @@ bpf_ctx_narrow_access_offset(u32 off, u32 size, u32 size_default)
 
 static inline void bpf_prog_lock_ro(struct bpf_prog *fp)
 {
-	set_vm_flush_reset_perms(fp);
-	set_memory_ro((unsigned long)fp, fp->pages);
+#ifndef CONFIG_BPF_JIT_ALWAYS_ON
+	if (!fp->jited) {
+		set_vm_flush_reset_perms(fp);
+		set_memory_ro((unsigned long)fp, fp->pages);
+	}
+#endif
 }
 
 static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr)
-- 
2.17.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-11-29 22:29 [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited Daniel Borkmann
@ 2019-11-30  1:37 ` Eric Dumazet
  2019-11-30  9:52   ` Daniel Borkmann
  0 siblings, 1 reply; 10+ messages in thread
From: Eric Dumazet @ 2019-11-30  1:37 UTC (permalink / raw)
  To: Daniel Borkmann, alexei.starovoitov; +Cc: peterz, netdev, bpf, H. Peter Anvin



On 11/29/19 2:29 PM, Daniel Borkmann wrote:
> For the case where the interpreter is compiled out or when the prog is jited
> it is completely unnecessary to set the BPF insn pages as read-only. In fact,
> on frequent churn of BPF programs, it could lead to performance degradation of
> the system over time since it would break the direct map down to 4k pages when
> calling set_memory_ro() for the insn buffer on x86-64 / arm64 and there is no
> reverse operation. Thus, avoid breaking up large pages for data maps, and only
> limit this to the module range used by the JIT where it is necessary to set
> the image read-only and executable.

Interesting... But why the non JIT case would need RO protection ?

Do you have any performance measures to share ?

Thanks.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-11-30  1:37 ` Eric Dumazet
@ 2019-11-30  9:52   ` Daniel Borkmann
  2019-12-01 17:54     ` Alexei Starovoitov
  2019-12-02  2:49     ` Eric Dumazet
  0 siblings, 2 replies; 10+ messages in thread
From: Daniel Borkmann @ 2019-11-30  9:52 UTC (permalink / raw)
  To: Eric Dumazet, alexei.starovoitov; +Cc: peterz, netdev, bpf, H. Peter Anvin

On 11/30/19 2:37 AM, Eric Dumazet wrote:
> On 11/29/19 2:29 PM, Daniel Borkmann wrote:
>> For the case where the interpreter is compiled out or when the prog is jited
>> it is completely unnecessary to set the BPF insn pages as read-only. In fact,
>> on frequent churn of BPF programs, it could lead to performance degradation of
>> the system over time since it would break the direct map down to 4k pages when
>> calling set_memory_ro() for the insn buffer on x86-64 / arm64 and there is no
>> reverse operation. Thus, avoid breaking up large pages for data maps, and only
>> limit this to the module range used by the JIT where it is necessary to set
>> the image read-only and executable.
> 
> Interesting... But why the non JIT case would need RO protection ?

It was done for interpreter around 5 years ago mainly due to concerns from security
folks that the BPF insn image could get corrupted (through some other bug in the
kernel) in post-verifier stage by an attacker and then there's nothing really that
would provide any sort of protection guarantees; pretty much the same reasons why
e.g. modules are set to read-only in the kernel.

> Do you have any performance measures to share ?

No numbers, and I'm also not aware of any reports from users, but it was recently
brought to our attention from mm folks during discussion of a different set:

https://lore.kernel.org/lkml/1572171452-7958-2-git-send-email-rppt@kernel.org/T/

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-11-30  9:52   ` Daniel Borkmann
@ 2019-12-01 17:54     ` Alexei Starovoitov
  2019-12-02  2:49     ` Eric Dumazet
  1 sibling, 0 replies; 10+ messages in thread
From: Alexei Starovoitov @ 2019-12-01 17:54 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Eric Dumazet, Peter Zijlstra, Network Development, bpf, H. Peter Anvin

On Sat, Nov 30, 2019 at 1:52 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 11/30/19 2:37 AM, Eric Dumazet wrote:
> > On 11/29/19 2:29 PM, Daniel Borkmann wrote:
> >> For the case where the interpreter is compiled out or when the prog is jited
> >> it is completely unnecessary to set the BPF insn pages as read-only. In fact,
> >> on frequent churn of BPF programs, it could lead to performance degradation of
> >> the system over time since it would break the direct map down to 4k pages when
> >> calling set_memory_ro() for the insn buffer on x86-64 / arm64 and there is no
> >> reverse operation. Thus, avoid breaking up large pages for data maps, and only
> >> limit this to the module range used by the JIT where it is necessary to set
> >> the image read-only and executable.
> >
> > Interesting... But why the non JIT case would need RO protection ?
>
> It was done for interpreter around 5 years ago mainly due to concerns from security
> folks that the BPF insn image could get corrupted (through some other bug in the
> kernel) in post-verifier stage by an attacker and then there's nothing really that
> would provide any sort of protection guarantees; pretty much the same reasons why
> e.g. modules are set to read-only in the kernel.
>
> > Do you have any performance measures to share ?
>
> No numbers, and I'm also not aware of any reports from users, but it was recently
> brought to our attention from mm folks during discussion of a different set:
>
> https://lore.kernel.org/lkml/1572171452-7958-2-git-send-email-rppt@kernel.org/T/

Applied. Thanks

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-11-30  9:52   ` Daniel Borkmann
  2019-12-01 17:54     ` Alexei Starovoitov
@ 2019-12-02  2:49     ` Eric Dumazet
  2019-12-02  3:44       ` hpa
  2019-12-02  8:30       ` Peter Zijlstra
  1 sibling, 2 replies; 10+ messages in thread
From: Eric Dumazet @ 2019-12-02  2:49 UTC (permalink / raw)
  To: Daniel Borkmann, alexei.starovoitov; +Cc: peterz, netdev, bpf, H. Peter Anvin



On 11/30/19 1:52 AM, Daniel Borkmann wrote:
> On 11/30/19 2:37 AM, Eric Dumazet wrote:
>> On 11/29/19 2:29 PM, Daniel Borkmann wrote:
>>> For the case where the interpreter is compiled out or when the prog is jited
>>> it is completely unnecessary to set the BPF insn pages as read-only. In fact,
>>> on frequent churn of BPF programs, it could lead to performance degradation of
>>> the system over time since it would break the direct map down to 4k pages when
>>> calling set_memory_ro() for the insn buffer on x86-64 / arm64 and there is no
>>> reverse operation. Thus, avoid breaking up large pages for data maps, and only
>>> limit this to the module range used by the JIT where it is necessary to set
>>> the image read-only and executable.
>>
>> Interesting... But why the non JIT case would need RO protection ?
> 
> It was done for interpreter around 5 years ago mainly due to concerns from security
> folks that the BPF insn image could get corrupted (through some other bug in the
> kernel) in post-verifier stage by an attacker and then there's nothing really that
> would provide any sort of protection guarantees; pretty much the same reasons why
> e.g. modules are set to read-only in the kernel.
> 
>> Do you have any performance measures to share ?
> 
> No numbers, and I'm also not aware of any reports from users, but it was recently
> brought to our attention from mm folks during discussion of a different set:
> 
> https://lore.kernel.org/lkml/1572171452-7958-2-git-send-email-rppt@kernel.org/T/
> 

Thanks for the link !

Having RO protection as a debug feature would be useful.

I believe we have CONFIG_STRICT_MODULE_RWX (and CONFIG_STRICT_KERNEL_RWX) for that already.

Or are we saying we also want to get rid of them ?

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-12-02  2:49     ` Eric Dumazet
@ 2019-12-02  3:44       ` hpa
  2019-12-02  8:30       ` Peter Zijlstra
  1 sibling, 0 replies; 10+ messages in thread
From: hpa @ 2019-12-02  3:44 UTC (permalink / raw)
  To: Eric Dumazet, Daniel Borkmann, alexei.starovoitov; +Cc: peterz, netdev, bpf

On December 1, 2019 6:49:32 PM PST, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>
>On 11/30/19 1:52 AM, Daniel Borkmann wrote:
>> On 11/30/19 2:37 AM, Eric Dumazet wrote:
>>> On 11/29/19 2:29 PM, Daniel Borkmann wrote:
>>>> For the case where the interpreter is compiled out or when the prog
>is jited
>>>> it is completely unnecessary to set the BPF insn pages as
>read-only. In fact,
>>>> on frequent churn of BPF programs, it could lead to performance
>degradation of
>>>> the system over time since it would break the direct map down to 4k
>pages when
>>>> calling set_memory_ro() for the insn buffer on x86-64 / arm64 and
>there is no
>>>> reverse operation. Thus, avoid breaking up large pages for data
>maps, and only
>>>> limit this to the module range used by the JIT where it is
>necessary to set
>>>> the image read-only and executable.
>>>
>>> Interesting... But why the non JIT case would need RO protection ?
>> 
>> It was done for interpreter around 5 years ago mainly due to concerns
>from security
>> folks that the BPF insn image could get corrupted (through some other
>bug in the
>> kernel) in post-verifier stage by an attacker and then there's
>nothing really that
>> would provide any sort of protection guarantees; pretty much the same
>reasons why
>> e.g. modules are set to read-only in the kernel.
>> 
>>> Do you have any performance measures to share ?
>> 
>> No numbers, and I'm also not aware of any reports from users, but it
>was recently
>> brought to our attention from mm folks during discussion of a
>different set:
>> 
>>
>https://lore.kernel.org/lkml/1572171452-7958-2-git-send-email-rppt@kernel.org/T/
>> 
>
>Thanks for the link !
>
>Having RO protection as a debug feature would be useful.
>
>I believe we have CONFIG_STRICT_MODULE_RWX (and
>CONFIG_STRICT_KERNEL_RWX) for that already.
>
>Or are we saying we also want to get rid of them ?

The notion is that for security there should never been a page which is both writable and executable at the same time. This makes it harder to inject code.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-12-02  2:49     ` Eric Dumazet
  2019-12-02  3:44       ` hpa
@ 2019-12-02  8:30       ` Peter Zijlstra
  2019-12-02  9:17         ` Daniel Borkmann
  1 sibling, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2019-12-02  8:30 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Daniel Borkmann, alexei.starovoitov, netdev, bpf, H. Peter Anvin

On Sun, Dec 01, 2019 at 06:49:32PM -0800, Eric Dumazet wrote:

> Thanks for the link !
> 
> Having RO protection as a debug feature would be useful.
> 
> I believe we have CONFIG_STRICT_MODULE_RWX (and CONFIG_STRICT_KERNEL_RWX) for that already.
> 
> Or are we saying we also want to get rid of them ?

No, in fact I'm working on making that stronger. We currently still have
a few cases that violate the W^X rule.

The thing is, when the BPF stuff is JIT'ed, the actual BPF instruction
page is not actually executed at all, so making it RO serves no purpose,
other than to fragment the direct map.

All actual code lives in the 2G range that x86_64 can directly branch
to, but this BPF instruction stuff lives in the general data heap and
can thus cause much more fragmentation of the direct map.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-12-02  8:30       ` Peter Zijlstra
@ 2019-12-02  9:17         ` Daniel Borkmann
  2019-12-02 16:19           ` Alexei Starovoitov
  0 siblings, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2019-12-02  9:17 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Eric Dumazet, alexei.starovoitov, netdev, bpf, H. Peter Anvin

On Mon, Dec 02, 2019 at 09:30:06AM +0100, Peter Zijlstra wrote:
> On Sun, Dec 01, 2019 at 06:49:32PM -0800, Eric Dumazet wrote:
> 
> > Thanks for the link !
> > 
> > Having RO protection as a debug feature would be useful.
> > 
> > I believe we have CONFIG_STRICT_MODULE_RWX (and CONFIG_STRICT_KERNEL_RWX) for that already.
> > 
> > Or are we saying we also want to get rid of them ?
> 
> No, in fact I'm working on making that stronger. We currently still have
> a few cases that violate the W^X rule.
> 
> The thing is, when the BPF stuff is JIT'ed, the actual BPF instruction
> page is not actually executed at all, so making it RO serves no purpose,
> other than to fragment the direct map.

Yes exactly, in that case it is only used for dumping the BPF insns back
to user space and therefore no need at all to set it RO. (The JITed image
however *is* set as RO. - Perhaps there was some confusion given your
earlier question.)

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-12-02  9:17         ` Daniel Borkmann
@ 2019-12-02 16:19           ` Alexei Starovoitov
  2019-12-02 20:09             ` Daniel Borkmann
  0 siblings, 1 reply; 10+ messages in thread
From: Alexei Starovoitov @ 2019-12-02 16:19 UTC (permalink / raw)
  To: Daniel Borkmann
  Cc: Peter Zijlstra, Eric Dumazet, Network Development, bpf, H. Peter Anvin

On Mon, Dec 2, 2019 at 1:17 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On Mon, Dec 02, 2019 at 09:30:06AM +0100, Peter Zijlstra wrote:
> > On Sun, Dec 01, 2019 at 06:49:32PM -0800, Eric Dumazet wrote:
> >
> > > Thanks for the link !
> > >
> > > Having RO protection as a debug feature would be useful.
> > >
> > > I believe we have CONFIG_STRICT_MODULE_RWX (and CONFIG_STRICT_KERNEL_RWX) for that already.
> > >
> > > Or are we saying we also want to get rid of them ?
> >
> > No, in fact I'm working on making that stronger. We currently still have
> > a few cases that violate the W^X rule.
> >
> > The thing is, when the BPF stuff is JIT'ed, the actual BPF instruction
> > page is not actually executed at all, so making it RO serves no purpose,
> > other than to fragment the direct map.
>
> Yes exactly, in that case it is only used for dumping the BPF insns back
> to user space and therefore no need at all to set it RO. (The JITed image
> however *is* set as RO. - Perhaps there was some confusion given your
> earlier question.)

May be we should also flip the default to net.core.bpf_jit_enable=1
for x86-64 ? and may be arm64 ? These two JITs are well tested
and maintained.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited
  2019-12-02 16:19           ` Alexei Starovoitov
@ 2019-12-02 20:09             ` Daniel Borkmann
  0 siblings, 0 replies; 10+ messages in thread
From: Daniel Borkmann @ 2019-12-02 20:09 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: Peter Zijlstra, Eric Dumazet, Network Development, bpf, H. Peter Anvin

On Mon, Dec 02, 2019 at 08:19:45AM -0800, Alexei Starovoitov wrote:
> On Mon, Dec 2, 2019 at 1:17 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > On Mon, Dec 02, 2019 at 09:30:06AM +0100, Peter Zijlstra wrote:
> > > On Sun, Dec 01, 2019 at 06:49:32PM -0800, Eric Dumazet wrote:
> > >
> > > > Thanks for the link !
> > > >
> > > > Having RO protection as a debug feature would be useful.
> > > >
> > > > I believe we have CONFIG_STRICT_MODULE_RWX (and CONFIG_STRICT_KERNEL_RWX) for that already.
> > > >
> > > > Or are we saying we also want to get rid of them ?
> > >
> > > No, in fact I'm working on making that stronger. We currently still have
> > > a few cases that violate the W^X rule.
> > >
> > > The thing is, when the BPF stuff is JIT'ed, the actual BPF instruction
> > > page is not actually executed at all, so making it RO serves no purpose,
> > > other than to fragment the direct map.
> >
> > Yes exactly, in that case it is only used for dumping the BPF insns back
> > to user space and therefore no need at all to set it RO. (The JITed image
> > however *is* set as RO. - Perhaps there was some confusion given your
> > earlier question.)
> 
> May be we should also flip the default to net.core.bpf_jit_enable=1
> for x86-64 ? and may be arm64 ? These two JITs are well tested
> and maintained.

Seems reasonable given their status and exposure they've had over the years. I
can follow-up on that.

Thanks,
Daniel

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, back to index

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-29 22:29 [PATCH bpf] bpf: avoid setting bpf insns pages read-only when prog is jited Daniel Borkmann
2019-11-30  1:37 ` Eric Dumazet
2019-11-30  9:52   ` Daniel Borkmann
2019-12-01 17:54     ` Alexei Starovoitov
2019-12-02  2:49     ` Eric Dumazet
2019-12-02  3:44       ` hpa
2019-12-02  8:30       ` Peter Zijlstra
2019-12-02  9:17         ` Daniel Borkmann
2019-12-02 16:19           ` Alexei Starovoitov
2019-12-02 20:09             ` Daniel Borkmann

BPF Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/bpf/0 bpf/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 bpf bpf/ https://lore.kernel.org/bpf \
		bpf@vger.kernel.org
	public-inbox-index bpf

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.bpf


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git