[PATCH 0/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[PATCH 0/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Yanfei Xu]

Hi,

When looking at the IPI counts in /proc/interrupts, there are always 
IPI1 happened on isolated cpus, even if the cpus had been idle with 
nohz. However, we should bother these cpus as less as possible.

The IPI1 were raised by flush_icache_range in bpf_int_jit_compile(). 
Futher, the calling of it was introduced in 3b8c9f1cdfc5("arm64: 
IPI each CPU after invalidating the I-cache for kernel mappings"), 
then I found the bpf case seems no need this operation. But I'm not 
sure, and still learning the JIT codes meanwhile. If I am wrong, 
please fix me, many thanks!

Yanfei Xu (1):
  bpf: avoid unnecessary IPI in bpf_flush_icache

 arch/arm64/net/bpf_jit_comp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.27.0

[PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Yanfei Xu]

It's no need to trigger IPI for keeping pipeline fresh in bpf case.

Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
---
 arch/arm64/net/bpf_jit_comp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index f7b194878a99..5311f8be4ba4 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -974,7 +974,7 @@ static int validate_code(struct jit_ctx *ctx)
 
 static inline void bpf_flush_icache(void *start, void *end)
 {
-	flush_icache_range((unsigned long)start, (unsigned long)end);
+	__flush_icache_range((unsigned long)start, (unsigned long)end);
 }
 
 struct arm64_jit_data {
-- 
2.27.0

Re: [PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Daniel Borkmann]

On 6/1/21 5:06 PM, Yanfei Xu wrote:
> It's no need to trigger IPI for keeping pipeline fresh in bpf case.

This needs a more concrete explanation/analysis on "why it is safe" to do so
rather than just saying that it is not needed.

> Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
> ---
>   arch/arm64/net/bpf_jit_comp.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index f7b194878a99..5311f8be4ba4 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -974,7 +974,7 @@ static int validate_code(struct jit_ctx *ctx)
>   
>   static inline void bpf_flush_icache(void *start, void *end)
>   {
> -	flush_icache_range((unsigned long)start, (unsigned long)end);
> +	__flush_icache_range((unsigned long)start, (unsigned long)end);
>   }
>   
>   struct arm64_jit_data {
> 

Re: [PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Will Deacon]

On Tue, Jun 01, 2021 at 07:20:04PM +0200, Daniel Borkmann wrote:
> On 6/1/21 5:06 PM, Yanfei Xu wrote:
> > It's no need to trigger IPI for keeping pipeline fresh in bpf case.
> 
> This needs a more concrete explanation/analysis on "why it is safe" to do so
> rather than just saying that it is not needed.

Agreed. You need to show how the executing thread ends up going through a
context synchronizing operation before jumping to the generated code if
the IPI here is removed.

Will

Re: [PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Xu, Yanfei]

On 6/2/21 1:41 AM, Will Deacon wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> On Tue, Jun 01, 2021 at 07:20:04PM +0200, Daniel Borkmann wrote:
>> On 6/1/21 5:06 PM, Yanfei Xu wrote:
>>> It's no need to trigger IPI for keeping pipeline fresh in bpf case.
>>
>> This needs a more concrete explanation/analysis on "why it is safe" to do so
>> rather than just saying that it is not needed.
> 
> Agreed. You need to show how the executing thread ends up going through a
> context synchronizing operation before jumping to the generated code if
> the IPI here is removed.

This patch came out with I looked through ftrace codes. Ftrace modify
the text code and don't send IPI in aarch64_insn_patch_text_nosync(). I
mistakenly thought the bpf is same with ftrace.

But now I'm still not sure why the ftrace don't need the IPI to go
through context synchronizing, maybe the worst situation is omit a
tracing event?

Thanks,
Yanfei

> 
> Will
> 

Re: [PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Will Deacon]

On Wed, Jun 02, 2021 at 07:26:03PM +0800, Xu, Yanfei wrote:
> 
> 
> On 6/2/21 1:41 AM, Will Deacon wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> > 
> > On Tue, Jun 01, 2021 at 07:20:04PM +0200, Daniel Borkmann wrote:
> > > On 6/1/21 5:06 PM, Yanfei Xu wrote:
> > > > It's no need to trigger IPI for keeping pipeline fresh in bpf case.
> > > 
> > > This needs a more concrete explanation/analysis on "why it is safe" to do so
> > > rather than just saying that it is not needed.
> > 
> > Agreed. You need to show how the executing thread ends up going through a
> > context synchronizing operation before jumping to the generated code if
> > the IPI here is removed.
> 
> This patch came out with I looked through ftrace codes. Ftrace modify
> the text code and don't send IPI in aarch64_insn_patch_text_nosync(). I
> mistakenly thought the bpf is same with ftrace.
> 
> But now I'm still not sure why the ftrace don't need the IPI to go
> through context synchronizing, maybe the worst situation is omit a
> tracing event?

I think ftrace handles this itself via ftrace_sync_ipi, no?

Will

Re: [PATCH 1/1] bpf: avoid unnecessary IPI in bpf_flush_icache

[Xu, Yanfei]

On 6/3/21 1:26 AM, Will Deacon wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> On Wed, Jun 02, 2021 at 07:26:03PM +0800, Xu, Yanfei wrote:
>>
>>
>> On 6/2/21 1:41 AM, Will Deacon wrote:
>>> [Please note: This e-mail is from an EXTERNAL e-mail address]
>>>
>>> On Tue, Jun 01, 2021 at 07:20:04PM +0200, Daniel Borkmann wrote:
>>>> On 6/1/21 5:06 PM, Yanfei Xu wrote:
>>>>> It's no need to trigger IPI for keeping pipeline fresh in bpf case.
>>>>
>>>> This needs a more concrete explanation/analysis on "why it is safe" to do so
>>>> rather than just saying that it is not needed.
>>>
>>> Agreed. You need to show how the executing thread ends up going through a
>>> context synchronizing operation before jumping to the generated code if
>>> the IPI here is removed.
>>
>> This patch came out with I looked through ftrace codes. Ftrace modify
>> the text code and don't send IPI in aarch64_insn_patch_text_nosync(). I
>> mistakenly thought the bpf is same with ftrace.
>>
>> But now I'm still not sure why the ftrace don't need the IPI to go
>> through context synchronizing, maybe the worst situation is omit a
>> tracing event?
> 
> I think ftrace handles this itself via ftrace_sync_ipi, no?

Ah, Yes! I missed this...

Anyway, thanks for your reminding and very sorry for my noise.

Regards,
Yanfei

> 
> Will
>