* [PATCH 0/3] kasan: memorize and print call_rcu stack @ 2020-05-06 5:18 Walter Wu 2020-05-06 5:53 ` Qian Cai 0 siblings, 1 reply; 6+ messages in thread From: Walter Wu @ 2020-05-06 5:18 UTC (permalink / raw) To: Andrey Ryabinin, Alexander Potapenko, Dmitry Vyukov, Matthias Brugger, Paul E . McKenney, Josh Triplett, Mathieu Desnoyers, Lai Jiangshan, Joel Fernandes, Andrew Morton Cc: Walter Wu, wsd_upstream, linux-kernel, kasan-dev, linux-mm, linux-mediatek, linux-arm-kernel This patchset improves KASAN reports by making them to have call_rcu() call stack information. It is helpful for programmers to solve use-after-free or double-free memory issue. The KASAN report was as follows(cleaned up slightly): BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 Freed by task 0: save_stack+0x24/0x50 __kasan_slab_free+0x110/0x178 kasan_slab_free+0x10/0x18 kfree+0x98/0x270 kasan_rcu_reclaim+0x1c/0x60 rcu_core+0x8b4/0x10f8 rcu_core_si+0xc/0x18 efi_header_end+0x238/0xa6c First call_rcu() call stack: save_stack+0x24/0x50 kasan_record_callrcu+0xc8/0xd8 call_rcu+0x190/0x580 kasan_rcu_uaf+0x1d8/0x278 Last call_rcu() call stack: (stack is not available) Add new CONFIG option to record first and last call_rcu() call stack and KASAN report prints two call_rcu() call stack. This option doesn't increase the cost of memory consumption. It is only suitable for generic KASAN. [1]https://bugzilla.kernel.org/show_bug.cgi?id=198437 Walter Wu (3): rcu/kasan: record and print call_rcu() call stack kasan: record and print the free track kasan: add KASAN_RCU_STACK_RECORD documentation Documentation/dev-tools/kasan.rst | 21 +++++++++++++++++++++ include/linux/kasan.h | 7 +++++++ kernel/rcu/tree.c | 5 +++++ lib/Kconfig.kasan | 11 +++++++++++ mm/kasan/common.c | 31 +++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 12 ++++++++++++ mm/kasan/report.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++------- 7 files changed, 133 insertions(+), 7 deletions(-) _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] kasan: memorize and print call_rcu stack 2020-05-06 5:18 [PATCH 0/3] kasan: memorize and print call_rcu stack Walter Wu @ 2020-05-06 5:53 ` Qian Cai 2020-05-06 6:23 ` Walter Wu 0 siblings, 1 reply; 6+ messages in thread From: Qian Cai @ 2020-05-06 5:53 UTC (permalink / raw) To: Walter Wu Cc: wsd_upstream, Paul E . McKenney, linux-mm, Lai Jiangshan, Josh Triplett, kasan-dev, linux-kernel, Joel Fernandes, linux-mediatek, Alexander Potapenko, linux-arm-kernel, Matthias Brugger, Andrey Ryabinin, Andrew Morton, Dmitry Vyukov, Mathieu Desnoyers > On May 6, 2020, at 1:19 AM, Walter Wu <walter-zh.wu@mediatek.com> wrote: > > This patchset improves KASAN reports by making them to have > call_rcu() call stack information. It is helpful for programmers > to solve use-after-free or double-free memory issue. > > The KASAN report was as follows(cleaned up slightly): > > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 > > Freed by task 0: > save_stack+0x24/0x50 > __kasan_slab_free+0x110/0x178 > kasan_slab_free+0x10/0x18 > kfree+0x98/0x270 > kasan_rcu_reclaim+0x1c/0x60 > rcu_core+0x8b4/0x10f8 > rcu_core_si+0xc/0x18 > efi_header_end+0x238/0xa6c > > First call_rcu() call stack: > save_stack+0x24/0x50 > kasan_record_callrcu+0xc8/0xd8 > call_rcu+0x190/0x580 > kasan_rcu_uaf+0x1d8/0x278 > > Last call_rcu() call stack: > (stack is not available) > > > Add new CONFIG option to record first and last call_rcu() call stack > and KASAN report prints two call_rcu() call stack. > > This option doesn't increase the cost of memory consumption. It is > only suitable for generic KASAN. I don’t understand why this needs to be a Kconfig option at all. If call_rcu() stacks are useful in general, then just always gather those information. How do developers judge if they need to select this option or not? _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] kasan: memorize and print call_rcu stack 2020-05-06 5:53 ` Qian Cai @ 2020-05-06 6:23 ` Walter Wu 2020-05-06 9:37 ` Dmitry Vyukov 0 siblings, 1 reply; 6+ messages in thread From: Walter Wu @ 2020-05-06 6:23 UTC (permalink / raw) To: Qian Cai Cc: wsd_upstream, Paul E . McKenney, linux-mm, Lai Jiangshan, Josh Triplett, kasan-dev, linux-kernel, Joel Fernandes, linux-mediatek, Alexander Potapenko, linux-arm-kernel, Matthias Brugger, Andrey Ryabinin, Andrew Morton, Dmitry Vyukov, Mathieu Desnoyers On Wed, 2020-05-06 at 01:53 -0400, Qian Cai wrote: > > > On May 6, 2020, at 1:19 AM, Walter Wu <walter-zh.wu@mediatek.com> wrote: > > > > This patchset improves KASAN reports by making them to have > > call_rcu() call stack information. It is helpful for programmers > > to solve use-after-free or double-free memory issue. > > > > The KASAN report was as follows(cleaned up slightly): > > > > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 > > > > Freed by task 0: > > save_stack+0x24/0x50 > > __kasan_slab_free+0x110/0x178 > > kasan_slab_free+0x10/0x18 > > kfree+0x98/0x270 > > kasan_rcu_reclaim+0x1c/0x60 > > rcu_core+0x8b4/0x10f8 > > rcu_core_si+0xc/0x18 > > efi_header_end+0x238/0xa6c > > > > First call_rcu() call stack: > > save_stack+0x24/0x50 > > kasan_record_callrcu+0xc8/0xd8 > > call_rcu+0x190/0x580 > > kasan_rcu_uaf+0x1d8/0x278 > > > > Last call_rcu() call stack: > > (stack is not available) > > > > > > Add new CONFIG option to record first and last call_rcu() call stack > > and KASAN report prints two call_rcu() call stack. > > > > This option doesn't increase the cost of memory consumption. It is > > only suitable for generic KASAN. > > I don’t understand why this needs to be a Kconfig option at all. If call_rcu() stacks are useful in general, then just always gather those information. How do developers judge if they need to select this option or not? Because we don't want to increase slub meta-data size, so enabling this option can print call_rcu() stacks, but the in-use slub object doesn't print free stack. So if have out-of-bound issue, then it will not print free stack. It is a trade-off, see [1]. [1] https://bugzilla.kernel.org/show_bug.cgi?id=198437 Thanks _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] kasan: memorize and print call_rcu stack 2020-05-06 6:23 ` Walter Wu @ 2020-05-06 9:37 ` Dmitry Vyukov 2020-05-06 12:01 ` Walter Wu 0 siblings, 1 reply; 6+ messages in thread From: Dmitry Vyukov @ 2020-05-06 9:37 UTC (permalink / raw) To: Walter Wu Cc: Qian Cai, wsd_upstream, Paul E . McKenney, Linux-MM, Lai Jiangshan, Josh Triplett, kasan-dev, LKML, Joel Fernandes, linux-mediatek, Alexander Potapenko, Matthias Brugger, Andrey Ryabinin, Andrew Morton, Linux ARM, Mathieu Desnoyers On Wed, May 6, 2020 at 8:23 AM Walter Wu <walter-zh.wu@mediatek.com> wrote: > > > This patchset improves KASAN reports by making them to have > > > call_rcu() call stack information. It is helpful for programmers > > > to solve use-after-free or double-free memory issue. > > > > > > The KASAN report was as follows(cleaned up slightly): > > > > > > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 > > > > > > Freed by task 0: > > > save_stack+0x24/0x50 > > > __kasan_slab_free+0x110/0x178 > > > kasan_slab_free+0x10/0x18 > > > kfree+0x98/0x270 > > > kasan_rcu_reclaim+0x1c/0x60 > > > rcu_core+0x8b4/0x10f8 > > > rcu_core_si+0xc/0x18 > > > efi_header_end+0x238/0xa6c > > > > > > First call_rcu() call stack: > > > save_stack+0x24/0x50 > > > kasan_record_callrcu+0xc8/0xd8 > > > call_rcu+0x190/0x580 > > > kasan_rcu_uaf+0x1d8/0x278 > > > > > > Last call_rcu() call stack: > > > (stack is not available) > > > > > > > > > Add new CONFIG option to record first and last call_rcu() call stack > > > and KASAN report prints two call_rcu() call stack. > > > > > > This option doesn't increase the cost of memory consumption. It is > > > only suitable for generic KASAN. > > > > I don’t understand why this needs to be a Kconfig option at all. If call_rcu() stacks are useful in general, then just always gather those information. How do developers judge if they need to select this option or not? > > Because we don't want to increase slub meta-data size, so enabling this > option can print call_rcu() stacks, but the in-use slub object doesn't > print free stack. So if have out-of-bound issue, then it will not print > free stack. It is a trade-off, see [1]. > > [1] https://bugzilla.kernel.org/show_bug.cgi?id=198437 Hi Walter, Great you are tackling this! I have the same general sentiment as Qian. I would enable this unconditionally because: 1. We still can't get both rcu stack and free stack. I would assume most kernel testing systems need to enable this (we definitely enable on syzbot). This means we do not have free stack for allocation objects in any reports coming from testing systems. Which greatly diminishes the value of the other mode. 2. Kernel is undertested. Introducing any additional configuration options is a problem in such context. Chances are that some of the modes are not working or will break in future. 3. That free stack actually causes lots of confusion and I never found it useful: https://bugzilla.kernel.org/show_bug.cgi?id=198425 If it's a very delayed UAF, either one may get another report for the same bug with not so delayed UAF, or if it's way too delayed, then the previous free stack is wrong as well. 4. Most users don't care that much about debugging tools to learn every bit of every debugging tool and spend time fine-tuning it for their context. Most KASAN users won't even be aware of this choice, and they will just use whatever is the default. 5. Each configuration option increases implementation complexity. What would have value is if we figure out how to make both of them work at the same time without increasing memory consumption. But I don't see any way to do this. I propose to make this the only mode. I am sure lots of users will find this additional stack useful, whereas the free stack is even frequently confusing. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] kasan: memorize and print call_rcu stack 2020-05-06 9:37 ` Dmitry Vyukov @ 2020-05-06 12:01 ` Walter Wu 2020-05-06 12:16 ` Dmitry Vyukov 0 siblings, 1 reply; 6+ messages in thread From: Walter Wu @ 2020-05-06 12:01 UTC (permalink / raw) To: Dmitry Vyukov Cc: Qian Cai, wsd_upstream, Paul E . McKenney, Linux-MM, Lai Jiangshan, Josh Triplett, kasan-dev, LKML, Joel Fernandes, linux-mediatek, Alexander Potapenko, Matthias Brugger, Andrey Ryabinin, Andrew Morton, Linux ARM, Mathieu Desnoyers On Wed, 2020-05-06 at 11:37 +0200, 'Dmitry Vyukov' via kasan-dev wrote: > On Wed, May 6, 2020 at 8:23 AM Walter Wu <walter-zh.wu@mediatek.com> wrote: > > > > This patchset improves KASAN reports by making them to have > > > > call_rcu() call stack information. It is helpful for programmers > > > > to solve use-after-free or double-free memory issue. > > > > > > > > The KASAN report was as follows(cleaned up slightly): > > > > > > > > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 > > > > > > > > Freed by task 0: > > > > save_stack+0x24/0x50 > > > > __kasan_slab_free+0x110/0x178 > > > > kasan_slab_free+0x10/0x18 > > > > kfree+0x98/0x270 > > > > kasan_rcu_reclaim+0x1c/0x60 > > > > rcu_core+0x8b4/0x10f8 > > > > rcu_core_si+0xc/0x18 > > > > efi_header_end+0x238/0xa6c > > > > > > > > First call_rcu() call stack: > > > > save_stack+0x24/0x50 > > > > kasan_record_callrcu+0xc8/0xd8 > > > > call_rcu+0x190/0x580 > > > > kasan_rcu_uaf+0x1d8/0x278 > > > > > > > > Last call_rcu() call stack: > > > > (stack is not available) > > > > > > > > > > > > Add new CONFIG option to record first and last call_rcu() call stack > > > > and KASAN report prints two call_rcu() call stack. > > > > > > > > This option doesn't increase the cost of memory consumption. It is > > > > only suitable for generic KASAN. > > > > > > I don’t understand why this needs to be a Kconfig option at all. If call_rcu() stacks are useful in general, then just always gather those information. How do developers judge if they need to select this option or not? > > > > Because we don't want to increase slub meta-data size, so enabling this > > option can print call_rcu() stacks, but the in-use slub object doesn't > > print free stack. So if have out-of-bound issue, then it will not print > > free stack. It is a trade-off, see [1]. > > > > [1] https://bugzilla.kernel.org/show_bug.cgi?id=198437 > > Hi Walter, > > Great you are tackling this! > > I have the same general sentiment as Qian. I would enable this > unconditionally because: > > 1. We still can't get both rcu stack and free stack. I would assume > most kernel testing systems need to enable this (we definitely enable > on syzbot). This means we do not have free stack for allocation > objects in any reports coming from testing systems. Which greatly > diminishes the value of the other mode. > > 2. Kernel is undertested. Introducing any additional configuration > options is a problem in such context. Chances are that some of the > modes are not working or will break in future. > > 3. That free stack actually causes lots of confusion and I never found > it useful: > https://bugzilla.kernel.org/show_bug.cgi?id=198425 > If it's a very delayed UAF, either one may get another report for the > same bug with not so delayed UAF, or if it's way too delayed, then the > previous free stack is wrong as well. > > 4. Most users don't care that much about debugging tools to learn > every bit of every debugging tool and spend time fine-tuning it for > their context. Most KASAN users won't even be aware of this choice, > and they will just use whatever is the default. > > 5. Each configuration option increases implementation complexity. > > What would have value is if we figure out how to make both of them > work at the same time without increasing memory consumption. But I > don't see any way to do this. > > I propose to make this the only mode. I am sure lots of users will > find this additional stack useful, whereas the free stack is even > frequently confusing. > Ok. If we want to have a default enabling it, but it should only work in generic KASAN, because we need to get object status(allocation or freeing) from shadow memory, tag-based KASAN can't do it. So we should have a default enabling it in generic KASAN? _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] kasan: memorize and print call_rcu stack 2020-05-06 12:01 ` Walter Wu @ 2020-05-06 12:16 ` Dmitry Vyukov 0 siblings, 0 replies; 6+ messages in thread From: Dmitry Vyukov @ 2020-05-06 12:16 UTC (permalink / raw) To: Walter Wu Cc: Qian Cai, wsd_upstream, Paul E . McKenney, Linux-MM, Lai Jiangshan, Josh Triplett, kasan-dev, LKML, Joel Fernandes, linux-mediatek, Alexander Potapenko, Matthias Brugger, Andrey Ryabinin, Andrew Morton, Linux ARM, Mathieu Desnoyers On Wed, May 6, 2020 at 2:01 PM Walter Wu <walter-zh.wu@mediatek.com> wrote: > > On Wed, 2020-05-06 at 11:37 +0200, 'Dmitry Vyukov' via kasan-dev wrote: > > On Wed, May 6, 2020 at 8:23 AM Walter Wu <walter-zh.wu@mediatek.com> wrote: > > > > > This patchset improves KASAN reports by making them to have > > > > > call_rcu() call stack information. It is helpful for programmers > > > > > to solve use-after-free or double-free memory issue. > > > > > > > > > > The KASAN report was as follows(cleaned up slightly): > > > > > > > > > > BUG: KASAN: use-after-free in kasan_rcu_reclaim+0x58/0x60 > > > > > > > > > > Freed by task 0: > > > > > save_stack+0x24/0x50 > > > > > __kasan_slab_free+0x110/0x178 > > > > > kasan_slab_free+0x10/0x18 > > > > > kfree+0x98/0x270 > > > > > kasan_rcu_reclaim+0x1c/0x60 > > > > > rcu_core+0x8b4/0x10f8 > > > > > rcu_core_si+0xc/0x18 > > > > > efi_header_end+0x238/0xa6c > > > > > > > > > > First call_rcu() call stack: > > > > > save_stack+0x24/0x50 > > > > > kasan_record_callrcu+0xc8/0xd8 > > > > > call_rcu+0x190/0x580 > > > > > kasan_rcu_uaf+0x1d8/0x278 > > > > > > > > > > Last call_rcu() call stack: > > > > > (stack is not available) > > > > > > > > > > > > > > > Add new CONFIG option to record first and last call_rcu() call stack > > > > > and KASAN report prints two call_rcu() call stack. > > > > > > > > > > This option doesn't increase the cost of memory consumption. It is > > > > > only suitable for generic KASAN. > > > > > > > > I don’t understand why this needs to be a Kconfig option at all. If call_rcu() stacks are useful in general, then just always gather those information. How do developers judge if they need to select this option or not? > > > > > > Because we don't want to increase slub meta-data size, so enabling this > > > option can print call_rcu() stacks, but the in-use slub object doesn't > > > print free stack. So if have out-of-bound issue, then it will not print > > > free stack. It is a trade-off, see [1]. > > > > > > [1] https://bugzilla.kernel.org/show_bug.cgi?id=198437 > > > > Hi Walter, > > > > Great you are tackling this! > > > > I have the same general sentiment as Qian. I would enable this > > unconditionally because: > > > > 1. We still can't get both rcu stack and free stack. I would assume > > most kernel testing systems need to enable this (we definitely enable > > on syzbot). This means we do not have free stack for allocation > > objects in any reports coming from testing systems. Which greatly > > diminishes the value of the other mode. > > > > 2. Kernel is undertested. Introducing any additional configuration > > options is a problem in such context. Chances are that some of the > > modes are not working or will break in future. > > > > 3. That free stack actually causes lots of confusion and I never found > > it useful: > > https://bugzilla.kernel.org/show_bug.cgi?id=198425 > > If it's a very delayed UAF, either one may get another report for the > > same bug with not so delayed UAF, or if it's way too delayed, then the > > previous free stack is wrong as well. > > > > 4. Most users don't care that much about debugging tools to learn > > every bit of every debugging tool and spend time fine-tuning it for > > their context. Most KASAN users won't even be aware of this choice, > > and they will just use whatever is the default. > > > > 5. Each configuration option increases implementation complexity. > > > > What would have value is if we figure out how to make both of them > > work at the same time without increasing memory consumption. But I > > don't see any way to do this. > > > > I propose to make this the only mode. I am sure lots of users will > > find this additional stack useful, whereas the free stack is even > > frequently confusing. > > > > Ok. > If we want to have a default enabling it, but it should only work in > generic KASAN, because we need to get object status(allocation or > freeing) from shadow memory, tag-based KASAN can't do it. So we should > have a default enabling it in generic KASAN? Yes, let's do generic KASAN always memorizes rcu stack; tags KASAN never memorizes rcu stacks. No new configurations. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-05-06 12:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-05-06 5:18 [PATCH 0/3] kasan: memorize and print call_rcu stack Walter Wu 2020-05-06 5:53 ` Qian Cai 2020-05-06 6:23 ` Walter Wu 2020-05-06 9:37 ` Dmitry Vyukov 2020-05-06 12:01 ` Walter Wu 2020-05-06 12:16 ` Dmitry Vyukov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).