[BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Masami Hiramatsu]

Since copy_optimized_instructions() misses to update real RIP
address while copying several instructions to working buffer,
it adjusts RIP-relative instruction with wrong RIP address for
the 2nd and subsequent instructions.

This may break the kernel (like kernel freeze) because
probed instruction can refer a wrong data. For example,
putting kprobes on cpumask_next hit this bug.

cpumask_next is normally like below if CONFIG_CPUMASK_OFFSTACK=y
(in this case nr_cpumask_bits is an alias of nr_cpu_ids)

 <cpumask_next>:
       48 89 f0                mov    %rsi,%rax
       8b 35 7b fb e2 00       mov    0xe2fb7b(%rip),%esi
        # ffffffff82db9e64 <nr_cpu_ids>
       55                      push   %rbp
...

If we put a kprobe on it and optimized with jump, it becomes
like this.

	e9 95 7d 07 1e	jmpq   0xffffffffa000207a
	7b fb	jnp    0xffffffff81f8a2e2 <cpumask_next+2>
	e2 00	loop   0xffffffff81f8a2e9 <cpumask_next+9>
	55	push   %rbp

This shows first 2 "mov" instructions are copied to trampoline
buffer at 0xffffffffa000207a. Here is the disassembled result.
(skipped optprobe template instructions)

Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
	54	push   %rsp
...
	48 83 c4 08	add    $0x8,%rsp
	9d	popfq
	48 89 f0	mov    %rsi,%rax
	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
        # 0xffffffff82db9e67 <nr_cpu_ids+3>

As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
*nr_cpu_ids. This leads a kernel freeze because cpumask_next()
always returns 0 and for_each_cpu() never ended.

Fixing this by adding len correctly to real RIP address while
copying.

Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
Reported-by: Michael Rodin <michael@rodin.online>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: stable@vger.kernel.org
---
 arch/x86/kernel/kprobes/opt.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
index eaf02f2e7300..e92672b8b490 100644
--- a/arch/x86/kernel/kprobes/opt.c
+++ b/arch/x86/kernel/kprobes/opt.c
@@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
 	int len = 0, ret;
 
 	while (len < RELATIVEJUMP_SIZE) {
-		ret = __copy_instruction(dest + len, src + len, real, &insn);
+		ret = __copy_instruction(dest + len, src + len, real + len,
+			&insn);
 		if (!ret || !can_boost(&insn, src + len))
 			return -EINVAL;
 		len += ret;

Re: [BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Steven Rostedt]

On Fri, 24 Aug 2018 02:16:12 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
> 	54	push   %rsp
> ...
> 	48 83 c4 08	add    $0x8,%rsp
> 	9d	popfq
> 	48 89 f0	mov    %rsi,%rax
> 	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
>         # 0xffffffff82db9e67 <nr_cpu_ids+3>
> 
> As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
> *nr_cpu_ids. This leads a kernel freeze because cpumask_next()
> always returns 0 and for_each_cpu() never ended.

Ouch! Nice catch.

> 
> Fixing this by adding len correctly to real RIP address while
> copying.
> 
> Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
> Reported-by: Michael Rodin <michael@rodin.online>
> Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> Cc: stable@vger.kernel.org
> ---
>  arch/x86/kernel/kprobes/opt.c |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
> index eaf02f2e7300..e92672b8b490 100644
> --- a/arch/x86/kernel/kprobes/opt.c
> +++ b/arch/x86/kernel/kprobes/opt.c
> @@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
>  	int len = 0, ret;
>  
>  	while (len < RELATIVEJUMP_SIZE) {
> -		ret = __copy_instruction(dest + len, src + len, real, &insn);
> +		ret = __copy_instruction(dest + len, src + len, real + len,
> +			&insn);
>  		if (!ret || !can_boost(&insn, src + len))
>  			return -EINVAL;
>  		len += ret;

Looking at the change that broke this we have:

> -static int copy_optimized_instructions(u8 *dest, u8 *src)
> +static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
>  {
>         struct insn insn;
>         int len = 0, ret;
>  
>         while (len < RELATIVEJUMP_SIZE) {
> -               ret = __copy_instruction(dest + len, src + len, &insn);
> +               ret = __copy_instruction(dest + len, src + len, real, &insn);

Where "real" was added as a parameter to __copy_instruction. Note that
we pass in "dest + len" but not "real + len" as you patch fixes.
__copy_instruction was changed by the bad commit with:

> -int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
> +int __copy_instruction(u8 *dest, u8 *src, u8 *real, struct insn *insn)
>  {
>         kprobe_opcode_t buf[MAX_INSN_SIZE];
>         unsigned long recovered_insn =
> @@ -387,11 +388,11 @@ int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
>                  * have given.
>                  */
>                 newdisp = (u8 *) src + (s64) insn->displacement.value
> -                         - (u8 *) dest;
> +                         - (u8 *) real;

"real" replaces "dest", which was the first parameter to __copy_instruction.

>                         return 0;

And:

>  int arch_prepare_optimized_kprobe(struct optimized_kprobe *op,
>                                   struct kprobe *__unused)
>  {
> -       u8 *buf;
> -       int ret;
> +       u8 *buf = NULL, *slot;
> +       int ret, len;
>         long rel;
>  
>         if (!can_optimize((unsigned long)op->kp.addr))
>                 return -EILSEQ;
>  
> -       op->optinsn.insn = get_optinsn_slot();
> -       if (!op->optinsn.insn)
> +       buf = kzalloc(MAX_OPTINSN_SIZE, GFP_KERNEL);
> +       if (!buf)
>                 return -ENOMEM;
>  
> +       op->optinsn.insn = slot = get_optinsn_slot();
> +       if (!slot) {
> +               ret = -ENOMEM;
> +               goto out;
> +       }
> +
>         /*
>          * Verify if the address gap is in 2GB range, because this uses
>          * a relative jump.
>          */
> -       rel = (long)op->optinsn.insn - (long)op->kp.addr + RELATIVEJUMP_SIZE;
> +       rel = (long)slot - (long)op->kp.addr + RELATIVEJUMP_SIZE;
>         if (abs(rel) > 0x7fffffff) {
> -               __arch_remove_optimized_kprobe(op, 0);
> -               return -ERANGE;
> +               ret = -ERANGE;
> +               goto err;
>         }
>  
> -       buf = (u8 *)op->optinsn.insn;

"slot" is equivalent to the old "buf".

> -       set_memory_rw((unsigned long)buf & PAGE_MASK, 1);
> +       /* Copy arch-dep-instance from template */
> +       memcpy(buf, &optprobe_template_entry, TMPL_END_IDX);
>  
>         /* Copy instructions into the out-of-line buffer */
> -       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr);
> -       if (ret < 0) {
> -               __arch_remove_optimized_kprobe(op, 0);
> -               return ret;
> -       }
> +       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr,
> +                                         slot + TMPL_END_IDX);

We pass in "real" as "slot + TMPL_END_IDX" and "dest" as "buf +
TMPL_END_IDX", thus to make it be equivalent to the code before this
commit, "real" should have "+ len" added to it in order to be
equivalent to what was there before.

That said...

Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

-- Steve


> +       if (ret < 0)
> +               goto err;

Re: [BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Masami Hiramatsu]

On Thu, 23 Aug 2018 21:41:09 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Fri, 24 Aug 2018 02:16:12 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
> > 	54	push   %rsp
> > ...
> > 	48 83 c4 08	add    $0x8,%rsp
> > 	9d	popfq
> > 	48 89 f0	mov    %rsi,%rax
> > 	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
> >         # 0xffffffff82db9e67 <nr_cpu_ids+3>
> > 
> > As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
> > *nr_cpu_ids. This leads a kernel freeze because cpumask_next()
> > always returns 0 and for_each_cpu() never ended.
> 
> Ouch! Nice catch.
> 
> > 
> > Fixing this by adding len correctly to real RIP address while
> > copying.
> > 
> > Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
> > Reported-by: Michael Rodin <michael@rodin.online>
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > Cc: stable@vger.kernel.org
> > ---
> >  arch/x86/kernel/kprobes/opt.c |    3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
> > index eaf02f2e7300..e92672b8b490 100644
> > --- a/arch/x86/kernel/kprobes/opt.c
> > +++ b/arch/x86/kernel/kprobes/opt.c
> > @@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
> >  	int len = 0, ret;
> >  
> >  	while (len < RELATIVEJUMP_SIZE) {
> > -		ret = __copy_instruction(dest + len, src + len, real, &insn);
> > +		ret = __copy_instruction(dest + len, src + len, real + len,
> > +			&insn);
> >  		if (!ret || !can_boost(&insn, src + len))
> >  			return -EINVAL;
> >  		len += ret;
> 
> Looking at the change that broke this we have:
> 
> > -static int copy_optimized_instructions(u8 *dest, u8 *src)
> > +static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
> >  {
> >         struct insn insn;
> >         int len = 0, ret;
> >  
> >         while (len < RELATIVEJUMP_SIZE) {
> > -               ret = __copy_instruction(dest + len, src + len, &insn);
> > +               ret = __copy_instruction(dest + len, src + len, real, &insn);
> 
> Where "real" was added as a parameter to __copy_instruction. Note that
> we pass in "dest + len" but not "real + len" as you patch fixes.
> __copy_instruction was changed by the bad commit with:
> 
> > -int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
> > +int __copy_instruction(u8 *dest, u8 *src, u8 *real, struct insn *insn)
> >  {
> >         kprobe_opcode_t buf[MAX_INSN_SIZE];
> >         unsigned long recovered_insn =
> > @@ -387,11 +388,11 @@ int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
> >                  * have given.
> >                  */
> >                 newdisp = (u8 *) src + (s64) insn->displacement.value
> > -                         - (u8 *) dest;
> > +                         - (u8 *) real;
> 
> "real" replaces "dest", which was the first parameter to __copy_instruction.
> 
> >                         return 0;
> 
> And:
> 
> >  int arch_prepare_optimized_kprobe(struct optimized_kprobe *op,
> >                                   struct kprobe *__unused)
> >  {
> > -       u8 *buf;
> > -       int ret;
> > +       u8 *buf = NULL, *slot;
> > +       int ret, len;
> >         long rel;
> >  
> >         if (!can_optimize((unsigned long)op->kp.addr))
> >                 return -EILSEQ;
> >  
> > -       op->optinsn.insn = get_optinsn_slot();
> > -       if (!op->optinsn.insn)
> > +       buf = kzalloc(MAX_OPTINSN_SIZE, GFP_KERNEL);
> > +       if (!buf)
> >                 return -ENOMEM;
> >  
> > +       op->optinsn.insn = slot = get_optinsn_slot();
> > +       if (!slot) {
> > +               ret = -ENOMEM;
> > +               goto out;
> > +       }
> > +
> >         /*
> >          * Verify if the address gap is in 2GB range, because this uses
> >          * a relative jump.
> >          */
> > -       rel = (long)op->optinsn.insn - (long)op->kp.addr + RELATIVEJUMP_SIZE;
> > +       rel = (long)slot - (long)op->kp.addr + RELATIVEJUMP_SIZE;
> >         if (abs(rel) > 0x7fffffff) {
> > -               __arch_remove_optimized_kprobe(op, 0);
> > -               return -ERANGE;
> > +               ret = -ERANGE;
> > +               goto err;
> >         }
> >  
> > -       buf = (u8 *)op->optinsn.insn;
> 
> "slot" is equivalent to the old "buf".
> 
> > -       set_memory_rw((unsigned long)buf & PAGE_MASK, 1);
> > +       /* Copy arch-dep-instance from template */
> > +       memcpy(buf, &optprobe_template_entry, TMPL_END_IDX);
> >  
> >         /* Copy instructions into the out-of-line buffer */
> > -       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr);
> > -       if (ret < 0) {
> > -               __arch_remove_optimized_kprobe(op, 0);
> > -               return ret;
> > -       }
> > +       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr,
> > +                                         slot + TMPL_END_IDX);
> 
> We pass in "real" as "slot + TMPL_END_IDX" and "dest" as "buf +
> TMPL_END_IDX", thus to make it be equivalent to the code before this
> commit, "real" should have "+ len" added to it in order to be
> equivalent to what was there before.

Right! The broken commit splits trampoline buffer into 
"temporary" destination buffer and "real" trampoline buffer,
and use the "real" address for RIP-relative adjustment.
However, I forgot to introduce update the "real" address
in the copying loop.

> 
> That said...
> 
> Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

Thanks!


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Masami Hiramatsu]

Hi Ingo,

Could you pick this fix to urgent branch?

Thank you,

On Fri, 24 Aug 2018 16:57:19 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Thu, 23 Aug 2018 21:41:09 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > On Fri, 24 Aug 2018 02:16:12 +0900
> > Masami Hiramatsu <mhiramat@kernel.org> wrote:
> > 
> > > Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
> > > 	54	push   %rsp
> > > ...
> > > 	48 83 c4 08	add    $0x8,%rsp
> > > 	9d	popfq
> > > 	48 89 f0	mov    %rsi,%rax
> > > 	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
> > >         # 0xffffffff82db9e67 <nr_cpu_ids+3>
> > > 
> > > As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
> > > *nr_cpu_ids. This leads a kernel freeze because cpumask_next()
> > > always returns 0 and for_each_cpu() never ended.
> > 
> > Ouch! Nice catch.
> > 
> > > 
> > > Fixing this by adding len correctly to real RIP address while
> > > copying.
> > > 
> > > Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
> > > Reported-by: Michael Rodin <michael@rodin.online>
> > > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > > Cc: stable@vger.kernel.org
> > > ---
> > >  arch/x86/kernel/kprobes/opt.c |    3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
> > > index eaf02f2e7300..e92672b8b490 100644
> > > --- a/arch/x86/kernel/kprobes/opt.c
> > > +++ b/arch/x86/kernel/kprobes/opt.c
> > > @@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
> > >  	int len = 0, ret;
> > >  
> > >  	while (len < RELATIVEJUMP_SIZE) {
> > > -		ret = __copy_instruction(dest + len, src + len, real, &insn);
> > > +		ret = __copy_instruction(dest + len, src + len, real + len,
> > > +			&insn);
> > >  		if (!ret || !can_boost(&insn, src + len))
> > >  			return -EINVAL;
> > >  		len += ret;
> > 
> > Looking at the change that broke this we have:
> > 
> > > -static int copy_optimized_instructions(u8 *dest, u8 *src)
> > > +static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
> > >  {
> > >         struct insn insn;
> > >         int len = 0, ret;
> > >  
> > >         while (len < RELATIVEJUMP_SIZE) {
> > > -               ret = __copy_instruction(dest + len, src + len, &insn);
> > > +               ret = __copy_instruction(dest + len, src + len, real, &insn);
> > 
> > Where "real" was added as a parameter to __copy_instruction. Note that
> > we pass in "dest + len" but not "real + len" as you patch fixes.
> > __copy_instruction was changed by the bad commit with:
> > 
> > > -int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
> > > +int __copy_instruction(u8 *dest, u8 *src, u8 *real, struct insn *insn)
> > >  {
> > >         kprobe_opcode_t buf[MAX_INSN_SIZE];
> > >         unsigned long recovered_insn =
> > > @@ -387,11 +388,11 @@ int __copy_instruction(u8 *dest, u8 *src, struct insn *insn)
> > >                  * have given.
> > >                  */
> > >                 newdisp = (u8 *) src + (s64) insn->displacement.value
> > > -                         - (u8 *) dest;
> > > +                         - (u8 *) real;
> > 
> > "real" replaces "dest", which was the first parameter to __copy_instruction.
> > 
> > >                         return 0;
> > 
> > And:
> > 
> > >  int arch_prepare_optimized_kprobe(struct optimized_kprobe *op,
> > >                                   struct kprobe *__unused)
> > >  {
> > > -       u8 *buf;
> > > -       int ret;
> > > +       u8 *buf = NULL, *slot;
> > > +       int ret, len;
> > >         long rel;
> > >  
> > >         if (!can_optimize((unsigned long)op->kp.addr))
> > >                 return -EILSEQ;
> > >  
> > > -       op->optinsn.insn = get_optinsn_slot();
> > > -       if (!op->optinsn.insn)
> > > +       buf = kzalloc(MAX_OPTINSN_SIZE, GFP_KERNEL);
> > > +       if (!buf)
> > >                 return -ENOMEM;
> > >  
> > > +       op->optinsn.insn = slot = get_optinsn_slot();
> > > +       if (!slot) {
> > > +               ret = -ENOMEM;
> > > +               goto out;
> > > +       }
> > > +
> > >         /*
> > >          * Verify if the address gap is in 2GB range, because this uses
> > >          * a relative jump.
> > >          */
> > > -       rel = (long)op->optinsn.insn - (long)op->kp.addr + RELATIVEJUMP_SIZE;
> > > +       rel = (long)slot - (long)op->kp.addr + RELATIVEJUMP_SIZE;
> > >         if (abs(rel) > 0x7fffffff) {
> > > -               __arch_remove_optimized_kprobe(op, 0);
> > > -               return -ERANGE;
> > > +               ret = -ERANGE;
> > > +               goto err;
> > >         }
> > >  
> > > -       buf = (u8 *)op->optinsn.insn;
> > 
> > "slot" is equivalent to the old "buf".
> > 
> > > -       set_memory_rw((unsigned long)buf & PAGE_MASK, 1);
> > > +       /* Copy arch-dep-instance from template */
> > > +       memcpy(buf, &optprobe_template_entry, TMPL_END_IDX);
> > >  
> > >         /* Copy instructions into the out-of-line buffer */
> > > -       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr);
> > > -       if (ret < 0) {
> > > -               __arch_remove_optimized_kprobe(op, 0);
> > > -               return ret;
> > > -       }
> > > +       ret = copy_optimized_instructions(buf + TMPL_END_IDX, op->kp.addr,
> > > +                                         slot + TMPL_END_IDX);
> > 
> > We pass in "real" as "slot + TMPL_END_IDX" and "dest" as "buf +
> > TMPL_END_IDX", thus to make it be equivalent to the code before this
> > commit, "real" should have "+ len" added to it in order to be
> > equivalent to what was there before.
> 
> Right! The broken commit splits trampoline buffer into 
> "temporary" destination buffer and "real" trampoline buffer,
> and use the "real" address for RIP-relative adjustment.
> However, I forgot to introduce update the "real" address
> in the copying loop.
> 
> > 
> > That said...
> > 
> > Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> 
> Thanks!
> 
> 
> -- 
> Masami Hiramatsu <mhiramat@kernel.org>


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Ingo Molnar]

* Masami Hiramatsu <mhiramat@kernel.org> wrote:

> Since copy_optimized_instructions() misses to update real RIP
> address while copying several instructions to working buffer,
> it adjusts RIP-relative instruction with wrong RIP address for
> the 2nd and subsequent instructions.
> 
> This may break the kernel (like kernel freeze) because
> probed instruction can refer a wrong data. For example,
> putting kprobes on cpumask_next hit this bug.
> 
> cpumask_next is normally like below if CONFIG_CPUMASK_OFFSTACK=y
> (in this case nr_cpumask_bits is an alias of nr_cpu_ids)
> 
>  <cpumask_next>:
>        48 89 f0                mov    %rsi,%rax
>        8b 35 7b fb e2 00       mov    0xe2fb7b(%rip),%esi
>         # ffffffff82db9e64 <nr_cpu_ids>
>        55                      push   %rbp
> ...
> 
> If we put a kprobe on it and optimized with jump, it becomes
> like this.
> 
> 	e9 95 7d 07 1e	jmpq   0xffffffffa000207a
> 	7b fb	jnp    0xffffffff81f8a2e2 <cpumask_next+2>
> 	e2 00	loop   0xffffffff81f8a2e9 <cpumask_next+9>
> 	55	push   %rbp
> 
> This shows first 2 "mov" instructions are copied to trampoline
> buffer at 0xffffffffa000207a. Here is the disassembled result.
> (skipped optprobe template instructions)
> 
> Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
> 	54	push   %rsp
> ...
> 	48 83 c4 08	add    $0x8,%rsp
> 	9d	popfq
> 	48 89 f0	mov    %rsi,%rax
> 	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
>         # 0xffffffff82db9e67 <nr_cpu_ids+3>
> 
> As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
> *nr_cpu_ids. This leads a kernel freeze because cpumask_next()
> always returns 0 and for_each_cpu() never ended.
> 
> Fixing this by adding len correctly to real RIP address while
> copying.
> 
> Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
> Reported-by: Michael Rodin <michael@rodin.online>
> Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> Cc: stable@vger.kernel.org
> ---
>  arch/x86/kernel/kprobes/opt.c |    3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
> index eaf02f2e7300..e92672b8b490 100644
> --- a/arch/x86/kernel/kprobes/opt.c
> +++ b/arch/x86/kernel/kprobes/opt.c
> @@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
>  	int len = 0, ret;
>  
>  	while (len < RELATIVEJUMP_SIZE) {
> -		ret = __copy_instruction(dest + len, src + len, real, &insn);
> +		ret = __copy_instruction(dest + len, src + len, real + len,
> +			&insn);

I applied this, except that I unbroke the line: please ignore checkpatch 
in such cases where the cure is worse than the disease ...

I.e. even if slightly over 80 cols, this is more readable:

		ret = __copy_instruction(dest + len, src + len, real + len, &insn);

Thanks,

	Ingo

Re: [BUGFIX PATCH -tip] kprobes/x86: Fix to copy RIP relative instruction correctly

[Masami Hiramatsu]

On Tue, 4 Dec 2018 09:13:35 +0100
Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > Since copy_optimized_instructions() misses to update real RIP
> > address while copying several instructions to working buffer,
> > it adjusts RIP-relative instruction with wrong RIP address for
> > the 2nd and subsequent instructions.
> > 
> > This may break the kernel (like kernel freeze) because
> > probed instruction can refer a wrong data. For example,
> > putting kprobes on cpumask_next hit this bug.
> > 
> > cpumask_next is normally like below if CONFIG_CPUMASK_OFFSTACK=y
> > (in this case nr_cpumask_bits is an alias of nr_cpu_ids)
> > 
> >  <cpumask_next>:
> >        48 89 f0                mov    %rsi,%rax
> >        8b 35 7b fb e2 00       mov    0xe2fb7b(%rip),%esi
> >         # ffffffff82db9e64 <nr_cpu_ids>
> >        55                      push   %rbp
> > ...
> > 
> > If we put a kprobe on it and optimized with jump, it becomes
> > like this.
> > 
> > 	e9 95 7d 07 1e	jmpq   0xffffffffa000207a
> > 	7b fb	jnp    0xffffffff81f8a2e2 <cpumask_next+2>
> > 	e2 00	loop   0xffffffff81f8a2e9 <cpumask_next+9>
> > 	55	push   %rbp
> > 
> > This shows first 2 "mov" instructions are copied to trampoline
> > buffer at 0xffffffffa000207a. Here is the disassembled result.
> > (skipped optprobe template instructions)
> > 
> > Dump of assembler code from 0xffffffffa000207a to 0xffffffffa00020ea:
> > 	54	push   %rsp
> > ...
> > 	48 83 c4 08	add    $0x8,%rsp
> > 	9d	popfq
> > 	48 89 f0	mov    %rsi,%rax
> > 	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi
> >         # 0xffffffff82db9e67 <nr_cpu_ids+3>
> > 
> > As it shows, the 2nd mov accesses *(nr_cpu_ids+3) instead of
> > *nr_cpu_ids. This leads a kernel freeze because cpumask_next()
> > always returns 0 and for_each_cpu() never ended.
> > 
> > Fixing this by adding len correctly to real RIP address while
> > copying.
> > 
> > Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
> > Reported-by: Michael Rodin <michael@rodin.online>
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > Cc: stable@vger.kernel.org
> > ---
> >  arch/x86/kernel/kprobes/opt.c |    3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
> > index eaf02f2e7300..e92672b8b490 100644
> > --- a/arch/x86/kernel/kprobes/opt.c
> > +++ b/arch/x86/kernel/kprobes/opt.c
> > @@ -189,7 +189,8 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
> >  	int len = 0, ret;
> >  
> >  	while (len < RELATIVEJUMP_SIZE) {
> > -		ret = __copy_instruction(dest + len, src + len, real, &insn);
> > +		ret = __copy_instruction(dest + len, src + len, real + len,
> > +			&insn);
> 
> I applied this, except that I unbroke the line: please ignore checkpatch 
> in such cases where the cure is worse than the disease ...

Thanks Ingo!

> 
> I.e. even if slightly over 80 cols, this is more readable:
> 
> 		ret = __copy_instruction(dest + len, src + len, real + len, &insn);

Got it.

BTW, this ugry "+ len" repeat would be avoided. I would better make a new inline
function to wrap it.


Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

[tip:perf/urgent] kprobes/x86: Fix instruction patching corruption when copying more than one RIP-relative instruction

[tip-bot for Masami Hiramatsu]

Commit-ID:  43a1b0cb4cd6dbfd3cd9c10da663368394d299d8
Gitweb:     https://git.kernel.org/tip/43a1b0cb4cd6dbfd3cd9c10da663368394d299d8
Author:     Masami Hiramatsu <mhiramat@kernel.org>
AuthorDate: Fri, 24 Aug 2018 02:16:12 +0900
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 4 Dec 2018 09:35:20 +0100

kprobes/x86: Fix instruction patching corruption when copying more than one RIP-relative instruction

After copy_optimized_instructions() copies several instructions
to the working buffer it tries to fix up the real RIP address, but it
adjusts the RIP-relative instruction with an incorrect RIP address
for the 2nd and subsequent instructions due to a bug in the logic.

This will break the kernel pretty badly (with likely outcomes such as
a kernel freeze, a crash, or worse) because probed instructions can refer
to the wrong data.

For example putting kprobes on cpumask_next() typically hits this bug.

cpumask_next() is normally like below if CONFIG_CPUMASK_OFFSTACK=y
(in this case nr_cpumask_bits is an alias of nr_cpu_ids):

 <cpumask_next>:
	48 89 f0		mov    %rsi,%rax
	8b 35 7b fb e2 00	mov    0xe2fb7b(%rip),%esi # ffffffff82db9e64 <nr_cpu_ids>
	55			push   %rbp
...

If we put a kprobe on it and it gets jump-optimized, it gets
patched by the kprobes code like this:

 <cpumask_next>:
	e9 95 7d 07 1e		jmpq   0xffffffffa000207a
	7b fb			jnp    0xffffffff81f8a2e2 <cpumask_next+2>
	e2 00			loop   0xffffffff81f8a2e9 <cpumask_next+9>
	55			push   %rbp

This shows that the first two MOV instructions were copied to a
trampoline buffer at 0xffffffffa000207a.

Here is the disassembled result of the trampoline, skipping
the optprobe template instructions:

	# Dump of assembly code from 0xffffffffa000207a to 0xffffffffa00020ea:

	54			push   %rsp
	...
	48 83 c4 08		add    $0x8,%rsp
	9d			popfq
	48 89 f0		mov    %rsi,%rax
	8b 35 82 7d db e2	mov    -0x1d24827e(%rip),%esi # 0xffffffff82db9e67 <nr_cpu_ids+3>

This dump shows that the second MOV accesses *(nr_cpu_ids+3) instead of
the original *nr_cpu_ids. This leads to a kernel freeze because
cpumask_next() always returns 0 and for_each_cpu() never ends.

Fix this by adding 'len' correctly to the real RIP address while
copying.

[ mingo: Improved the changelog. ]

Reported-by: Michael Rodin <michael@rodin.online>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org # v4.15+
Fixes: 63fef14fc98a ("kprobes/x86: Make insn buffer always ROX and use text_poke()")
Link: http://lkml.kernel.org/r/153504457253.22602.1314289671019919596.stgit@devbox
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/kprobes/opt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c
index 40b16b270656..6adf6e6c2933 100644
--- a/arch/x86/kernel/kprobes/opt.c
+++ b/arch/x86/kernel/kprobes/opt.c
@@ -189,7 +189,7 @@ static int copy_optimized_instructions(u8 *dest, u8 *src, u8 *real)
 	int len = 0, ret;
 
 	while (len < RELATIVEJUMP_SIZE) {
-		ret = __copy_instruction(dest + len, src + len, real, &insn);
+		ret = __copy_instruction(dest + len, src + len, real + len, &insn);
 		if (!ret || !can_boost(&insn, src + len))
 			return -EINVAL;
 		len += ret;