Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Naveen N. Rao]

Akanksha J N wrote:
> Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
> arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
> ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
> set to NULL.
> Extend multiple kprobes test to add kprobes on first 256 bytes within a
> function, to be able to test potential issues with kprobes on
> successive instructions.
> The '|| true' is added with the echo statement to ignore errors that are
> caused by trying to add kprobes to non probeable lines and continue with
> the test.
> 
> Signed-off-by: Akanksha J N <akanksha@linux.ibm.com>
> ---
>  .../selftests/ftrace/test.d/kprobe/multiple_kprobes.tc        | 4 ++++
>  1 file changed, 4 insertions(+)

Thanks for adding this test!

> 
> diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> index be754f5bcf79..f005c2542baa 100644
> --- a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> +++ b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> @@ -25,6 +25,10 @@ if [ $L -ne 256 ]; then
>    exit_fail
>  fi
> 
> +for i in `seq 0 255`; do
> +  echo p $FUNCTION_FORK+${i} >> kprobe_events || true
> +done
> +
>  cat kprobe_events >> $testlog
> 
>  echo 1 > events/kprobes/enable

Thinking about this more, I wonder if we should add an explicit fork 
after enabling the events, similar to kprobe_args.tc:
	( echo "forked" )

That will ensure we hit all the probes we added. With that change:
Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>


- Naveen

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Masami Hiramatsu]

On Thu, 12 Jan 2023 18:51:14 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:

> Akanksha J N wrote:
> > Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
> > arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
> > ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
> > set to NULL.
> > Extend multiple kprobes test to add kprobes on first 256 bytes within a
> > function, to be able to test potential issues with kprobes on
> > successive instructions.

What is the purpose of that test? If you intended to add a kprobe events
with some offset so that it becomes ftrace-based kprobe, it should be
a different test case, because

 - This is a test case for checking multiple (at least 256) kprobe events
  can be defined and enabled.

 - If you want to check the ftrace-based kprobe, it should be near the
   function entry, maybe within 16 bytes or so.

 - Also, you don't need to enable it at once (and should not for this case).

> > The '|| true' is added with the echo statement to ignore errors that are
> > caused by trying to add kprobes to non probeable lines and continue with
> > the test.

Can you add another test case for that? (and send it to the MLs which Cc'd
to this mail)
e.g. 

   for i in `seq 0 16`; do
     echo p:testprobe $FUNCTION_FORK+${i} >> kprobe_events || continue
     echo 1 > events/kprobes/testprobe/enable
     ( echo "forked" )
     echo 0 > events/kprobes/testprobe/enable
     echo > kprobe_events
   done


BTW, after we introduce the fprobe event (https://lore.kernel.org/linux-trace-kernel/166792255429.919356.14116090269057513181.stgit@devnote3/) that test case may be
update to check fprobe events.

Thank you,

> > 
> > Signed-off-by: Akanksha J N <akanksha@linux.ibm.com>
> > ---
> >  .../selftests/ftrace/test.d/kprobe/multiple_kprobes.tc        | 4 ++++
> >  1 file changed, 4 insertions(+)
> 
> Thanks for adding this test!
> 
> > 
> > diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> > index be754f5bcf79..f005c2542baa 100644
> > --- a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> > +++ b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> > @@ -25,6 +25,10 @@ if [ $L -ne 256 ]; then
> >    exit_fail
> >  fi
> > 
> > +for i in `seq 0 255`; do
> > +  echo p $FUNCTION_FORK+${i} >> kprobe_events || true
> > +done
> > +
> >  cat kprobe_events >> $testlog
> > 
> >  echo 1 > events/kprobes/enable
> 
> Thinking about this more, I wonder if we should add an explicit fork 
> after enabling the events, similar to kprobe_args.tc:
> 	( echo "forked" )
> 
> That will ensure we hit all the probes we added. With that change:
> Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> 
> 
> - Naveen


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Naveen N. Rao]

Masami Hiramatsu wrote:
> On Thu, 12 Jan 2023 18:51:14 +0530
> "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> 
>> Akanksha J N wrote:
>> > Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
>> > arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
>> > ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
>> > set to NULL.
>> > Extend multiple kprobes test to add kprobes on first 256 bytes within a
>> > function, to be able to test potential issues with kprobes on
>> > successive instructions.
> 
> What is the purpose of that test? If you intended to add a kprobe events
> with some offset so that it becomes ftrace-based kprobe, it should be
> a different test case, because

This is a follow up to:
http://lore.kernel.org/1664530538.ke6dp49pwh.naveen@linux.ibm.com

The intent is to add consecutive probes covering KPROBES_ON_FTRACE, 
vanilla trap-based kprobes as well as optprobes to ensure all of those 
and their interactions are good.

> 
>  - This is a test case for checking multiple (at least 256) kprobe events
>   can be defined and enabled.
> 
>  - If you want to check the ftrace-based kprobe, it should be near the
>    function entry, maybe within 16 bytes or so.
> 
>  - Also, you don't need to enable it at once (and should not for this case).
> 
>> > The '|| true' is added with the echo statement to ignore errors that are
>> > caused by trying to add kprobes to non probeable lines and continue with
>> > the test.
> 
> Can you add another test case for that? (and send it to the MLs which Cc'd
> to this mail)
> e.g. 
> 
>    for i in `seq 0 16`; do
>      echo p:testprobe $FUNCTION_FORK+${i} >> kprobe_events || continue
>      echo 1 > events/kprobes/testprobe/enable
>      ( echo "forked" )
>      echo 0 > events/kprobes/testprobe/enable
>      echo > kprobe_events
>    done

The current test to add multiple kprobes within a function also falls 
under the purview of multiple_kprobes.tc, but it can be split into a 
separate multiple_kprobes_func.tc if you think that will be better.

> 
> 
> BTW, after we introduce the fprobe event (https://lore.kernel.org/linux-trace-kernel/166792255429.919356.14116090269057513181.stgit@devnote3/) that test case may be
> update to check fprobe events.

Indeed, I suppose that can be a separate test.


Thanks,
Naveen

> 
> Thank you,
> 
>> > 
>> > Signed-off-by: Akanksha J N <akanksha@linux.ibm.com>
>> > ---
>> >  .../selftests/ftrace/test.d/kprobe/multiple_kprobes.tc        | 4 ++++
>> >  1 file changed, 4 insertions(+)
>> 
>> Thanks for adding this test!
>> 
>> > 
>> > diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
>> > index be754f5bcf79..f005c2542baa 100644
>> > --- a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
>> > +++ b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
>> > @@ -25,6 +25,10 @@ if [ $L -ne 256 ]; then
>> >    exit_fail
>> >  fi
>> > 
>> > +for i in `seq 0 255`; do
>> > +  echo p $FUNCTION_FORK+${i} >> kprobe_events || true
>> > +done
>> > +
>> >  cat kprobe_events >> $testlog
>> > 
>> >  echo 1 > events/kprobes/enable
>> 
>> Thinking about this more, I wonder if we should add an explicit fork 
>> after enabling the events, similar to kprobe_args.tc:
>> 	( echo "forked" )
>> 
>> That will ensure we hit all the probes we added. With that change:
>> Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
>> 
>> 
>> - Naveen
> 
> 
> -- 
> Masami Hiramatsu (Google) <mhiramat@kernel.org>
> 

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Masami Hiramatsu]

Hi Naveen,

On Fri, 13 Jan 2023 14:59:51 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:

> Masami Hiramatsu wrote:
> > On Thu, 12 Jan 2023 18:51:14 +0530
> > "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> > 
> >> Akanksha J N wrote:
> >> > Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
> >> > arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
> >> > ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
> >> > set to NULL.
> >> > Extend multiple kprobes test to add kprobes on first 256 bytes within a
> >> > function, to be able to test potential issues with kprobes on
> >> > successive instructions.
> > 
> > What is the purpose of that test? If you intended to add a kprobe events
> > with some offset so that it becomes ftrace-based kprobe, it should be
> > a different test case, because
> 
> This is a follow up to:
> http://lore.kernel.org/1664530538.ke6dp49pwh.naveen@linux.ibm.com
> 
> The intent is to add consecutive probes covering KPROBES_ON_FTRACE, 
> vanilla trap-based kprobes as well as optprobes to ensure all of those 
> and their interactions are good.

Hmm, that should be implemented for each architecture with specific
knowledge instead of random offset, so that we can ensure the kprobe
is on ftrace/optimized or using trap. Also, it should check the
debugfs/kprobes/list file.

> 
> > 
> >  - This is a test case for checking multiple (at least 256) kprobe events
> >   can be defined and enabled.
> > 
> >  - If you want to check the ftrace-based kprobe, it should be near the
> >    function entry, maybe within 16 bytes or so.
> > 
> >  - Also, you don't need to enable it at once (and should not for this case).
> > 
> >> > The '|| true' is added with the echo statement to ignore errors that are
> >> > caused by trying to add kprobes to non probeable lines and continue with
> >> > the test.
> > 
> > Can you add another test case for that? (and send it to the MLs which Cc'd
> > to this mail)
> > e.g. 
> > 
> >    for i in `seq 0 16`; do
> >      echo p:testprobe $FUNCTION_FORK+${i} >> kprobe_events || continue
> >      echo 1 > events/kprobes/testprobe/enable
> >      ( echo "forked" )
> >      echo 0 > events/kprobes/testprobe/enable
> >      echo > kprobe_events
> >    done
> 
> The current test to add multiple kprobes within a function also falls 
> under the purview of multiple_kprobes.tc, but it can be split into a 
> separate multiple_kprobes_func.tc if you think that will be better.
> 

Yes, please make it separate, this test case is for checking whether
the ftrace can define/enable/disable multiple kprobe events. Not for
checking kprobe with different types, nor checking interactions among
different types of kprobes.

(BTW, if you want to test optprobe on x86, you can not put the probes
 within the jump instruction (+5 bytes). It will unoptimize existing
 optimized kprobe in that case)

And do you really need to run "multiple" kprobes at once?
I think what you need is 'kprobe_opt_types.tc'.

> > 
> > 
> > BTW, after we introduce the fprobe event (https://lore.kernel.org/linux-trace-kernel/166792255429.919356.14116090269057513181.stgit@devnote3/) that test case may be
> > update to check fprobe events.
> 
> Indeed, I suppose that can be a separate test.

Thank you,

> 
> 
> Thanks,
> Naveen
> 
> > 
> > Thank you,
> > 
> >> > 
> >> > Signed-off-by: Akanksha J N <akanksha@linux.ibm.com>
> >> > ---
> >> >  .../selftests/ftrace/test.d/kprobe/multiple_kprobes.tc        | 4 ++++
> >> >  1 file changed, 4 insertions(+)
> >> 
> >> Thanks for adding this test!
> >> 
> >> > 
> >> > diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> >> > index be754f5bcf79..f005c2542baa 100644
> >> > --- a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> >> > +++ b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc
> >> > @@ -25,6 +25,10 @@ if [ $L -ne 256 ]; then
> >> >    exit_fail
> >> >  fi
> >> > 
> >> > +for i in `seq 0 255`; do
> >> > +  echo p $FUNCTION_FORK+${i} >> kprobe_events || true
> >> > +done
> >> > +
> >> >  cat kprobe_events >> $testlog
> >> > 
> >> >  echo 1 > events/kprobes/enable
> >> 
> >> Thinking about this more, I wonder if we should add an explicit fork 
> >> after enabling the events, similar to kprobe_args.tc:
> >> 	( echo "forked" )
> >> 
> >> That will ensure we hit all the probes we added. With that change:
> >> Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> >> 
> >> 
> >> - Naveen
> > 
> > 
> > -- 
> > Masami Hiramatsu (Google) <mhiramat@kernel.org>
> > 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Naveen N. Rao]

Masami Hiramatsu wrote:
> Hi Naveen,
> 
> On Fri, 13 Jan 2023 14:59:51 +0530
> "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> 
>> Masami Hiramatsu wrote:
>> > On Thu, 12 Jan 2023 18:51:14 +0530
>> > "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
>> > 
>> >> Akanksha J N wrote:
>> >> > Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
>> >> > arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
>> >> > ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
>> >> > set to NULL.
>> >> > Extend multiple kprobes test to add kprobes on first 256 bytes within a
>> >> > function, to be able to test potential issues with kprobes on
>> >> > successive instructions.
>> > 
>> > What is the purpose of that test? If you intended to add a kprobe events
>> > with some offset so that it becomes ftrace-based kprobe, it should be
>> > a different test case, because
>> 
>> This is a follow up to:
>> http://lore.kernel.org/1664530538.ke6dp49pwh.naveen@linux.ibm.com
>> 
>> The intent is to add consecutive probes covering KPROBES_ON_FTRACE, 
>> vanilla trap-based kprobes as well as optprobes to ensure all of those 
>> and their interactions are good.
> 
> Hmm, that should be implemented for each architecture with specific
> knowledge instead of random offset, so that we can ensure the kprobe
> is on ftrace/optimized or using trap. Also, it should check the
> debugfs/kprobes/list file.

...

> 
>> 
>> > 
>> >  - This is a test case for checking multiple (at least 256) kprobe events
>> >   can be defined and enabled.
>> > 
>> >  - If you want to check the ftrace-based kprobe, it should be near the
>> >    function entry, maybe within 16 bytes or so.
>> > 
>> >  - Also, you don't need to enable it at once (and should not for this case).
>> > 
>> >> > The '|| true' is added with the echo statement to ignore errors that are
>> >> > caused by trying to add kprobes to non probeable lines and continue with
>> >> > the test.
>> > 
>> > Can you add another test case for that? (and send it to the MLs which Cc'd
>> > to this mail)
>> > e.g. 
>> > 
>> >    for i in `seq 0 16`; do
>> >      echo p:testprobe $FUNCTION_FORK+${i} >> kprobe_events || continue
>> >      echo 1 > events/kprobes/testprobe/enable
>> >      ( echo "forked" )
>> >      echo 0 > events/kprobes/testprobe/enable
>> >      echo > kprobe_events
>> >    done
>> 
>> The current test to add multiple kprobes within a function also falls 
>> under the purview of multiple_kprobes.tc, but it can be split into a 
>> separate multiple_kprobes_func.tc if you think that will be better.
>> 
> 
> Yes, please make it separate, this test case is for checking whether
> the ftrace can define/enable/disable multiple kprobe events. Not for
> checking kprobe with different types, nor checking interactions among
> different types of kprobes.
> 
> (BTW, if you want to test optprobe on x86, you can not put the probes
>  within the jump instruction (+5 bytes). It will unoptimize existing
>  optimized kprobe in that case)

Ok, I can see why we won't be able to optimize any of the probes on x86 
with this approach. But, we should be able to do so on powerpc and arm, 
the only other architectures supporting OPTPROBES at this time. For x86, 
we may have to extend the test to check kprobes/list.

Crucially, I think trying to place a probe at each byte can still 
exercize interactions across KPROBES_ON_FTRACE and normal kprobes, so 
this test is still a good start. In addition, we get to ensure that 
kprobes infrastructure is rejecting placing probes at non-instruction 
boundaries.

> 
> And do you really need to run "multiple" kprobes at once?
> I think what you need is 'kprobe_opt_types.tc'.

Yes, enabling those probes is a good stress test to ensure we are only 
accepting valid probe locations.

multiple_kprobe_types.tc ? :)


Thanks,
Naveen

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Masami Hiramatsu]

Hi Naveen,

On Mon, 16 Jan 2023 14:02:04 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:

> Masami Hiramatsu wrote:
> > Hi Naveen,
> > 
> > On Fri, 13 Jan 2023 14:59:51 +0530
> > "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> > 
> >> Masami Hiramatsu wrote:
> >> > On Thu, 12 Jan 2023 18:51:14 +0530
> >> > "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:
> >> > 
> >> >> Akanksha J N wrote:
> >> >> > Commit 97f88a3d723162 ("powerpc/kprobes: Fix null pointer reference in
> >> >> > arch_prepare_kprobe()") fixed a recent kernel oops that was caused as
> >> >> > ftrace-based kprobe does not generate kprobe::ainsn::insn and it gets
> >> >> > set to NULL.
> >> >> > Extend multiple kprobes test to add kprobes on first 256 bytes within a
> >> >> > function, to be able to test potential issues with kprobes on
> >> >> > successive instructions.
> >> > 
> >> > What is the purpose of that test? If you intended to add a kprobe events
> >> > with some offset so that it becomes ftrace-based kprobe, it should be
> >> > a different test case, because
> >> 
> >> This is a follow up to:
> >> http://lore.kernel.org/1664530538.ke6dp49pwh.naveen@linux.ibm.com
> >> 
> >> The intent is to add consecutive probes covering KPROBES_ON_FTRACE, 
> >> vanilla trap-based kprobes as well as optprobes to ensure all of those 
> >> and their interactions are good.
> > 
> > Hmm, that should be implemented for each architecture with specific
> > knowledge instead of random offset, so that we can ensure the kprobe
> > is on ftrace/optimized or using trap. Also, it should check the
> > debugfs/kprobes/list file.
> 
> ...
> 
> > 
> >> 
> >> > 
> >> >  - This is a test case for checking multiple (at least 256) kprobe events
> >> >   can be defined and enabled.
> >> > 
> >> >  - If you want to check the ftrace-based kprobe, it should be near the
> >> >    function entry, maybe within 16 bytes or so.
> >> > 
> >> >  - Also, you don't need to enable it at once (and should not for this case).
> >> > 
> >> >> > The '|| true' is added with the echo statement to ignore errors that are
> >> >> > caused by trying to add kprobes to non probeable lines and continue with
> >> >> > the test.
> >> > 
> >> > Can you add another test case for that? (and send it to the MLs which Cc'd
> >> > to this mail)
> >> > e.g. 
> >> > 
> >> >    for i in `seq 0 16`; do
> >> >      echo p:testprobe $FUNCTION_FORK+${i} >> kprobe_events || continue
> >> >      echo 1 > events/kprobes/testprobe/enable
> >> >      ( echo "forked" )
> >> >      echo 0 > events/kprobes/testprobe/enable
> >> >      echo > kprobe_events
> >> >    done
> >> 
> >> The current test to add multiple kprobes within a function also falls 
> >> under the purview of multiple_kprobes.tc, but it can be split into a 
> >> separate multiple_kprobes_func.tc if you think that will be better.
> >> 
> > 
> > Yes, please make it separate, this test case is for checking whether
> > the ftrace can define/enable/disable multiple kprobe events. Not for
> > checking kprobe with different types, nor checking interactions among
> > different types of kprobes.
> > 
> > (BTW, if you want to test optprobe on x86, you can not put the probes
> >  within the jump instruction (+5 bytes). It will unoptimize existing
> >  optimized kprobe in that case)
> 
> Ok, I can see why we won't be able to optimize any of the probes on x86 
> with this approach. But, we should be able to do so on powerpc and arm, 
> the only other architectures supporting OPTPROBES at this time. For x86, 
> we may have to extend the test to check kprobes/list.

Are there any instruction type specific limitation on those arch for
using optprobe? I guess the 'call' (branch with link register) will not
able to be optimized because it leaves the trampoline address on the
stack.

> 
> Crucially, I think trying to place a probe at each byte can still 
> exercize interactions across KPROBES_ON_FTRACE and normal kprobes, so 
> this test is still a good start. In addition, we get to ensure that 
> kprobes infrastructure is rejecting placing probes at non-instruction 
> boundaries.

The interfere between probes can be happen between kprobes and optprobe
(*only on x86*), but not with KPORBES_ON_FTRACE. The ftrace replaced NOP
will be handled as one instruction. 

> > And do you really need to run "multiple" kprobes at once?
> > I think what you need is 'kprobe_opt_types.tc'.
> 
> Yes, enabling those probes is a good stress test to ensure we are only 
> accepting valid probe locations.
> 
> multiple_kprobe_types.tc ? :)

Please don't mixed it with the concept of 'multiple' probe test.
It is different that
 - kprobes can put probes on each instruction boundary.
 - kprobes can allocate and enable multiple probes at the same time.

What the multiple_kprobes.tc tests is the latter one.
(This is the reason why it chooses different functions so as not to
 interfere with each other.)

Thank you,

> 
> 
> Thanks,
> Naveen
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Naveen N. Rao]

Hi Masami,

Masami Hiramatsu wrote:
>> > 
>> > Yes, please make it separate, this test case is for checking whether
>> > the ftrace can define/enable/disable multiple kprobe events. Not for
>> > checking kprobe with different types, nor checking interactions among
>> > different types of kprobes.
>> > 
>> > (BTW, if you want to test optprobe on x86, you can not put the probes
>> >  within the jump instruction (+5 bytes). It will unoptimize existing
>> >  optimized kprobe in that case)
>> 
>> Ok, I can see why we won't be able to optimize any of the probes on x86 
>> with this approach. But, we should be able to do so on powerpc and arm, 
>> the only other architectures supporting OPTPROBES at this time. For x86, 
>> we may have to extend the test to check kprobes/list.
> 
> Are there any instruction type specific limitation on those arch for
> using optprobe? I guess the 'call' (branch with link register) will not
> able to be optimized because it leaves the trampoline address on the
> stack.

Yes, at least on powerpc, we only optimize ALU instructions and do not 
optimize load/store instructions, among many others. This is the reason 
we try to put a probe uptil 256 offset into a function in the proposed 
test, which will almost certainly catch an instruction that can be 
optimized.

> 
>> 
>> Crucially, I think trying to place a probe at each byte can still 
>> exercize interactions across KPROBES_ON_FTRACE and normal kprobes, so 
>> this test is still a good start. In addition, we get to ensure that 
>> kprobes infrastructure is rejecting placing probes at non-instruction 
>> boundaries.
> 
> The interfere between probes can be happen between kprobes and optprobe
> (*only on x86*), but not with KPORBES_ON_FTRACE. The ftrace replaced NOP
> will be handled as one instruction. 

Yes.

> 
>> > And do you really need to run "multiple" kprobes at once?
>> > I think what you need is 'kprobe_opt_types.tc'.
>> 
>> Yes, enabling those probes is a good stress test to ensure we are only 
>> accepting valid probe locations.
>> 
>> multiple_kprobe_types.tc ? :)
> 
> Please don't mixed it with the concept of 'multiple' probe test.
> It is different that
>  - kprobes can put probes on each instruction boundary.
>  - kprobes can allocate and enable multiple probes at the same time.
> 
> What the multiple_kprobes.tc tests is the latter one.
> (This is the reason why it chooses different functions so as not to
>  interfere with each other.)

Ok, I was coming from the point of view that both tests end up 
installing "multiple" kprobes, but I do see your point.

How about adding two new tests:
1. The same test as has been proposed in this thread: trying to add a 
kprobe at every byte within $FUNCTION_FORK upto an offset of 256 bytes. 
We can probably call it kprobe_insn_boundary.tc
2. A new test to ensure we can add different kprobe types 
(kprobe_opt_types.tc). This test will need to enable and check if each 
probe has been optimized or not and needs arch-specific knowledge so 
that we can take care of x86.

Would that be ok?


Thanks,
Naveen

Re: [PATCH] selftests/ftrace: Extend multiple_kprobes.tc to add multiple consecutive probes in a function

[Masami Hiramatsu]

On Wed, 25 Jan 2023 12:39:36 +0530
"Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com> wrote:

> Hi Masami,
> 
> Masami Hiramatsu wrote:
> >> > 
> >> > Yes, please make it separate, this test case is for checking whether
> >> > the ftrace can define/enable/disable multiple kprobe events. Not for
> >> > checking kprobe with different types, nor checking interactions among
> >> > different types of kprobes.
> >> > 
> >> > (BTW, if you want to test optprobe on x86, you can not put the probes
> >> >  within the jump instruction (+5 bytes). It will unoptimize existing
> >> >  optimized kprobe in that case)
> >> 
> >> Ok, I can see why we won't be able to optimize any of the probes on x86 
> >> with this approach. But, we should be able to do so on powerpc and arm, 
> >> the only other architectures supporting OPTPROBES at this time. For x86, 
> >> we may have to extend the test to check kprobes/list.
> > 
> > Are there any instruction type specific limitation on those arch for
> > using optprobe? I guess the 'call' (branch with link register) will not
> > able to be optimized because it leaves the trampoline address on the
> > stack.
> 
> Yes, at least on powerpc, we only optimize ALU instructions and do not 
> optimize load/store instructions, among many others. This is the reason 
> we try to put a probe uptil 256 offset into a function in the proposed 
> test, which will almost certainly catch an instruction that can be 
> optimized.
> 
> > 
> >> 
> >> Crucially, I think trying to place a probe at each byte can still 
> >> exercize interactions across KPROBES_ON_FTRACE and normal kprobes, so 
> >> this test is still a good start. In addition, we get to ensure that 
> >> kprobes infrastructure is rejecting placing probes at non-instruction 
> >> boundaries.
> > 
> > The interfere between probes can be happen between kprobes and optprobe
> > (*only on x86*), but not with KPORBES_ON_FTRACE. The ftrace replaced NOP
> > will be handled as one instruction. 
> 
> Yes.
> 
> > 
> >> > And do you really need to run "multiple" kprobes at once?
> >> > I think what you need is 'kprobe_opt_types.tc'.
> >> 
> >> Yes, enabling those probes is a good stress test to ensure we are only 
> >> accepting valid probe locations.
> >> 
> >> multiple_kprobe_types.tc ? :)
> > 
> > Please don't mixed it with the concept of 'multiple' probe test.
> > It is different that
> >  - kprobes can put probes on each instruction boundary.
> >  - kprobes can allocate and enable multiple probes at the same time.
> > 
> > What the multiple_kprobes.tc tests is the latter one.
> > (This is the reason why it chooses different functions so as not to
> >  interfere with each other.)
> 
> Ok, I was coming from the point of view that both tests end up 
> installing "multiple" kprobes, but I do see your point.
> 
> How about adding two new tests:
> 1. The same test as has been proposed in this thread: trying to add a 
> kprobe at every byte within $FUNCTION_FORK upto an offset of 256 bytes. 
> We can probably call it kprobe_insn_boundary.tc

OK.

> 2. A new test to ensure we can add different kprobe types 
> (kprobe_opt_types.tc). This test will need to enable and check if each 
> probe has been optimized or not and needs arch-specific knowledge so 
> that we can take care of x86.

OK, this should be only for x86. 

> 
> Would that be ok?

Yes, this sounds good to me. 

Thank you!

> 
> 
> Thanks,
> Naveen
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>