[PATCH v7 0/4] kasan: memorize and print call_rcu stack

[PATCH v7 0/4] kasan: memorize and print call_rcu stack

[Walter Wu]

This patchset improves KASAN reports by making them to have
call_rcu() call stack information. It is useful for programmers
to solve use-after-free or double-free memory issue.

The KASAN report was as follows(cleaned up slightly):

BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60

Freed by task 0:
 kasan_save_stack+0x24/0x50
 kasan_set_track+0x24/0x38
 kasan_set_free_info+0x18/0x20
 __kasan_slab_free+0x10c/0x170
 kasan_slab_free+0x10/0x18
 kfree+0x98/0x270
 kasan_rcu_reclaim+0x1c/0x60

Last call_rcu():
 kasan_save_stack+0x24/0x50
 kasan_record_aux_stack+0xbc/0xd0
 call_rcu+0x8c/0x580
 kasan_rcu_uaf+0xf4/0xf8

Generic KASAN will record the last two call_rcu() call stacks and
print up to 2 call_rcu() call stacks in KASAN report. it is only
suitable for generic KASAN.

This feature considers the size of struct kasan_alloc_meta and
kasan_free_meta, we try to optimize the structure layout and size
, lets it get better memory consumption.

[1]https://bugzilla.kernel.org/show_bug.cgi?id=198437
[2]https://groups.google.com/forum/#!searchin/kasan-dev/better$20stack$20traces$20for$20rcu%7Csort:date/kasan-dev/KQsjT_88hDE/7rNUZprRBgAJ

Changes since v1:
- remove new config option, default enable it in generic KASAN
- test this feature in SLAB/SLUB, it is pass.
- modify macro to be more clearly
- modify documentation

Changes since v2:
- change recording from first/last to the last two call stacks
- move free track into kasan free meta
- init slab_free_meta on object slot creation
- modify documentation

Changes since v3:
- change variable name to be more clearly
- remove the redundant condition
- remove init free meta-data and increasing object condition

Changes since v4:
- add a macro KASAN_KMALLOC_FREETRACK in order to check whether
  print free stack
- change printing message
- remove descriptions in Kocong.kasan

Changes since v5:
- reuse print_stack() in print_track()

Changes since v6:
- fix typo
- renamed the variable name in testcase

Walter Wu (4):
rcu: kasan: record and print call_rcu() call stack
kasan: record and print the free track
kasan: add tests for call_rcu stack recording
kasan: update documentation for generic kasan

Documentation/dev-tools/kasan.rst |  3 +++
include/linux/kasan.h             |  2 ++
kernel/rcu/tree.c                 |  2 ++
lib/test_kasan.c                  | 30 ++++++++++++++++++++++++++++++
mm/kasan/common.c                 | 26 ++++----------------------
mm/kasan/generic.c                | 43 +++++++++++++++++++++++++++++++++++++++++++
mm/kasan/generic_report.c         |  1 +
mm/kasan/kasan.h                  | 23 +++++++++++++++++++++--
mm/kasan/quarantine.c             |  1 +
mm/kasan/report.c                 | 54 +++++++++++++++++++++++++++---------------------------
mm/kasan/tags.c                   | 37 +++++++++++++++++++++++++++++++++++++
11 files changed, 171 insertions(+), 51 deletions(-)

Re: [PATCH v7 0/4] kasan: memorize and print call_rcu stack

[Walter Wu]

On Mon, 2020-06-01 at 13:08 +0800, Walter Wu wrote:
> This patchset improves KASAN reports by making them to have
> call_rcu() call stack information. It is useful for programmers
> to solve use-after-free or double-free memory issue.
> 
> The KASAN report was as follows(cleaned up slightly):
> 
> BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60
> 
> Freed by task 0:
>  kasan_save_stack+0x24/0x50
>  kasan_set_track+0x24/0x38
>  kasan_set_free_info+0x18/0x20
>  __kasan_slab_free+0x10c/0x170
>  kasan_slab_free+0x10/0x18
>  kfree+0x98/0x270
>  kasan_rcu_reclaim+0x1c/0x60
> 
> Last call_rcu():
>  kasan_save_stack+0x24/0x50
>  kasan_record_aux_stack+0xbc/0xd0
>  call_rcu+0x8c/0x580
>  kasan_rcu_uaf+0xf4/0xf8
> 
> Generic KASAN will record the last two call_rcu() call stacks and
> print up to 2 call_rcu() call stacks in KASAN report. it is only
> suitable for generic KASAN.
> 
> This feature considers the size of struct kasan_alloc_meta and
> kasan_free_meta, we try to optimize the structure layout and size
> , lets it get better memory consumption.
> 
> [1]https://bugzilla.kernel.org/show_bug.cgi?id=198437
> [2]https://groups.google.com/forum/#!searchin/kasan-dev/better$20stack$20traces$20for$20rcu%7Csort:date/kasan-dev/KQsjT_88hDE/7rNUZprRBgAJ
> 
> Changes since v1:
> - remove new config option, default enable it in generic KASAN
> - test this feature in SLAB/SLUB, it is pass.
> - modify macro to be more clearly
> - modify documentation
> 
> Changes since v2:
> - change recording from first/last to the last two call stacks
> - move free track into kasan free meta
> - init slab_free_meta on object slot creation
> - modify documentation
> 
> Changes since v3:
> - change variable name to be more clearly
> - remove the redundant condition
> - remove init free meta-data and increasing object condition
> 
> Changes since v4:
> - add a macro KASAN_KMALLOC_FREETRACK in order to check whether
>   print free stack
> - change printing message
> - remove descriptions in Kocong.kasan
> 
> Changes since v5:
> - reuse print_stack() in print_track()
> 
> Changes since v6:
> - fix typo
> - renamed the variable name in testcase
> 
> Walter Wu (4):
> rcu: kasan: record and print call_rcu() call stack
> kasan: record and print the free track
> kasan: add tests for call_rcu stack recording
> kasan: update documentation for generic kasan
> 

Hi Andrew,

Would you tell me why don't pick up this patches?
Do I miss something?

I will want to implement another new patches, but it need to depend on
this patches.


Thanks for your helps.

Walter

> Documentation/dev-tools/kasan.rst |  3 +++
> include/linux/kasan.h             |  2 ++
> kernel/rcu/tree.c                 |  2 ++
> lib/test_kasan.c                  | 30 ++++++++++++++++++++++++++++++
> mm/kasan/common.c                 | 26 ++++----------------------
> mm/kasan/generic.c                | 43 +++++++++++++++++++++++++++++++++++++++++++
> mm/kasan/generic_report.c         |  1 +
> mm/kasan/kasan.h                  | 23 +++++++++++++++++++++--
> mm/kasan/quarantine.c             |  1 +
> mm/kasan/report.c                 | 54 +++++++++++++++++++++++++++---------------------------
> mm/kasan/tags.c                   | 37 +++++++++++++++++++++++++++++++++++++
> 11 files changed, 171 insertions(+), 51 deletions(-)

Re: [PATCH v7 0/4] kasan: memorize and print call_rcu stack

[Dmitry Vyukov]

On Tue, Jun 23, 2020 at 10:09 AM Walter Wu <walter-zh.wu@mediatek.com> wrote:
>
> On Mon, 2020-06-01 at 13:08 +0800, Walter Wu wrote:
> > This patchset improves KASAN reports by making them to have
> > call_rcu() call stack information. It is useful for programmers
> > to solve use-after-free or double-free memory issue.
> >
> > The KASAN report was as follows(cleaned up slightly):
> >
> > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60
> >
> > Freed by task 0:
> >  kasan_save_stack+0x24/0x50
> >  kasan_set_track+0x24/0x38
> >  kasan_set_free_info+0x18/0x20
> >  __kasan_slab_free+0x10c/0x170
> >  kasan_slab_free+0x10/0x18
> >  kfree+0x98/0x270
> >  kasan_rcu_reclaim+0x1c/0x60
> >
> > Last call_rcu():
> >  kasan_save_stack+0x24/0x50
> >  kasan_record_aux_stack+0xbc/0xd0
> >  call_rcu+0x8c/0x580
> >  kasan_rcu_uaf+0xf4/0xf8
> >
> > Generic KASAN will record the last two call_rcu() call stacks and
> > print up to 2 call_rcu() call stacks in KASAN report. it is only
> > suitable for generic KASAN.
> >
> > This feature considers the size of struct kasan_alloc_meta and
> > kasan_free_meta, we try to optimize the structure layout and size
> > , lets it get better memory consumption.
> >
> > [1]https://bugzilla.kernel.org/show_bug.cgi?id=198437
> > [2]https://groups.google.com/forum/#!searchin/kasan-dev/better$20stack$20traces$20for$20rcu%7Csort:date/kasan-dev/KQsjT_88hDE/7rNUZprRBgAJ
> >
> > Changes since v1:
> > - remove new config option, default enable it in generic KASAN
> > - test this feature in SLAB/SLUB, it is pass.
> > - modify macro to be more clearly
> > - modify documentation
> >
> > Changes since v2:
> > - change recording from first/last to the last two call stacks
> > - move free track into kasan free meta
> > - init slab_free_meta on object slot creation
> > - modify documentation
> >
> > Changes since v3:
> > - change variable name to be more clearly
> > - remove the redundant condition
> > - remove init free meta-data and increasing object condition
> >
> > Changes since v4:
> > - add a macro KASAN_KMALLOC_FREETRACK in order to check whether
> >   print free stack
> > - change printing message
> > - remove descriptions in Kocong.kasan
> >
> > Changes since v5:
> > - reuse print_stack() in print_track()
> >
> > Changes since v6:
> > - fix typo
> > - renamed the variable name in testcase
> >
> > Walter Wu (4):
> > rcu: kasan: record and print call_rcu() call stack
> > kasan: record and print the free track
> > kasan: add tests for call_rcu stack recording
> > kasan: update documentation for generic kasan
> >
>
> Hi Andrew,
>
> Would you tell me why don't pick up this patches?
> Do I miss something?
>
> I will want to implement another new patches, but it need to depend on
> this patches.

On a related note.
Doing this for workqueue on top of these patches may be useful as
well, here is syzbot UAFs that mention process_one_work:
https://groups.google.com/forum/#!searchin/syzkaller-bugs/%22use-after-free%22$20process_one_work%7Csort:date

In some of these access/allocation happened in in process_one_work, in
some workqueue queueing stack may not add much.
But if we take the last one:
https://groups.google.com/forum/#!searchin/syzkaller-bugs/%22use-after-free%22$20process_one_work%7Csort:date/syzkaller-bugs/IYE0kt0BZMQ/zNM5rlzjAQAJ
It's exactly the same "free stack is useless" situation:

Freed by task 17:
 kfree+0x10a/0x220 mm/slab.c:3757
 process_one_work+0x76e/0xfd0 kernel/workqueue.c:2268
 worker_thread+0xa7f/0x1450 kernel/workqueue.c:2414
 kthread+0x353/0x380 kernel/kthread.c:268

The same may stand for times, I think I've seen some bugs where the
bad access happens in the timer as well.
Adding workqueue and timers should be pretty minimal change I think.


> > Documentation/dev-tools/kasan.rst |  3 +++
> > include/linux/kasan.h             |  2 ++
> > kernel/rcu/tree.c                 |  2 ++
> > lib/test_kasan.c                  | 30 ++++++++++++++++++++++++++++++
> > mm/kasan/common.c                 | 26 ++++----------------------
> > mm/kasan/generic.c                | 43 +++++++++++++++++++++++++++++++++++++++++++
> > mm/kasan/generic_report.c         |  1 +
> > mm/kasan/kasan.h                  | 23 +++++++++++++++++++++--
> > mm/kasan/quarantine.c             |  1 +
> > mm/kasan/report.c                 | 54 +++++++++++++++++++++++++++---------------------------
> > mm/kasan/tags.c                   | 37 +++++++++++++++++++++++++++++++++++++
> > 11 files changed, 171 insertions(+), 51 deletions(-)
>
> --
> You received this message because you are subscribed to the Google Groups "kasan-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/1592899732.13735.8.camel%40mtksdccf07.