From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka <vbabka@suse.cz> Cc: kernel test robot <oliver.sang@intel.com>, Oliver Glitta <glittao@gmail.com>, lkp@lists.01.org, lkp@intel.com, LKML <linux-kernel@vger.kernel.org>, linux-mm@kvack.org, Mike Rapoport <rppt@kernel.org> Subject: Re: [mm/slub] ae107fa919: BUG:unable_to_handle_page_fault_for_address Date: Sat, 12 Mar 2022 11:04:30 +0000 [thread overview] Message-ID: <Yix+Pn9a9yKIiXC4@ip-172-31-19-208.ap-northeast-1.compute.internal> (raw) In-Reply-To: <a8dc8bc4-9a00-b2f7-1f68-273f7a14a14c@suse.cz> On Sat, Mar 12, 2022 at 10:21:25AM +0100, Vlastimil Babka wrote: > On 3/12/22 02:10, Hyeonggon Yoo wrote: > > On Fri, Mar 11, 2022 at 04:46:00PM +0000, Hyeonggon Yoo wrote: > >> On Fri, Mar 11, 2022 at 04:36:47PM +0100, Vlastimil Babka wrote: > >>> On 3/11/22 15:54, Hyeonggon Yoo wrote: > >>>> On Wed, Mar 09, 2022 at 10:15:31AM +0800, kernel test robot wrote: > >>>>> > >>>>> > >>>>> Greeting, > >>>>> > >>>>> FYI, we noticed the following commit (built with gcc-9): > >>>>> > >>>>> commit: ae107fa91914f098cd54ab77e68f83dd6259e901 ("mm/slub: use stackdepot to save stack trace in objects") > >>>>> https://git.kernel.org/cgit/linux/kernel/git/vbabka/linux.git slub-stackdepot-v3r0 > >>>>> > >>>>> in testcase: boot > >>>>> > >>>>> on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G > >>>>> > >>>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): > >>>>> > >>>> > >>>> [+Cc Vlastimil and linux-mm] > >>> > >>> Thanks. > >>> lkp folks: it would be nice if I was CC'd automatically on this, it's a > >>> commit from my git tree and with by s-o-b :) > >>> > >>>> I _strongly_ suspect that this is because we don't initialize > >>>> stack_table[i] = NULL when we allocate it from memblock_alloc(). > >>> > >>> No, Mike (CC'd) suggested to drop the array init cycle, because > >>> memblock_alloc would zero the area anyway. > >> > >> Ah, you are right. My mistake. > >> > >>> There has to be a different > >>> reason. Wondering if dmesg contains the stack depot initialization message > >>> at all... > >> > >> I think I found the reason. > >> This is because of CONFIG_SLUB_DEBUG_ON. > >> It can enable debugging without passing boot parameter. > >> > >> if CONFIG_SLUB_DEBUG_ON=y && slub_debug is not passed, we do not call > >> stack_depot_want_early_init(), but the debugging flags are set. > >> > >> And we only call stack_depot_init() later in kmem_cache_create_usercopy(). > >> > >> so it crashed while creating boot cache. > > > > I tested this, and this was the reason. > > It crashed on CONFIG_SLUB_DEBUG_ON=y because stackdepot always assume > > that it was initialized in boot step, or failed > > (stack_depot_disable=true). > > > > But as it didn't even tried to initialize it, stack_table == NULL && > > stack_depot_disable == false. So accessing *(NULL + <hash value>) > > Thanks for finding the cause! > ;) > > Ideas? implementing something like kmem_cache_init_early() again? > > I think we could simply make CONFIG_SLUB_DEBUG_ON select/depend on > STACKDEPOT_ALWAYS_INIT? Oh, sounds better. If we make CONFIG_SLUB_DEBUG_ON select STACK_DEPOT_ALWAYS_INIT, that is simple solution. but stackdepot will be initialized on slub_debug=- too. But I think no one will set CONFIG_SLUB_DEBUG_ON=y if not debugging... I don't think making CONFIG_SLUB_DEBUG_ON depend on CONFIG_STACKDEPOT_ALWAYS_INIT is good solution. only KASAN selects it. -- Thank you, You are awesome! Hyeonggon :-)
WARNING: multiple messages have this Message-ID (diff)
From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: lkp@lists.01.org Subject: Re: [mm/slub] ae107fa919: BUG:unable_to_handle_page_fault_for_address Date: Sat, 12 Mar 2022 11:04:30 +0000 [thread overview] Message-ID: <Yix+Pn9a9yKIiXC4@ip-172-31-19-208.ap-northeast-1.compute.internal> (raw) In-Reply-To: <a8dc8bc4-9a00-b2f7-1f68-273f7a14a14c@suse.cz> [-- Attachment #1: Type: text/plain, Size: 3049 bytes --] On Sat, Mar 12, 2022 at 10:21:25AM +0100, Vlastimil Babka wrote: > On 3/12/22 02:10, Hyeonggon Yoo wrote: > > On Fri, Mar 11, 2022 at 04:46:00PM +0000, Hyeonggon Yoo wrote: > >> On Fri, Mar 11, 2022 at 04:36:47PM +0100, Vlastimil Babka wrote: > >>> On 3/11/22 15:54, Hyeonggon Yoo wrote: > >>>> On Wed, Mar 09, 2022 at 10:15:31AM +0800, kernel test robot wrote: > >>>>> > >>>>> > >>>>> Greeting, > >>>>> > >>>>> FYI, we noticed the following commit (built with gcc-9): > >>>>> > >>>>> commit: ae107fa91914f098cd54ab77e68f83dd6259e901 ("mm/slub: use stackdepot to save stack trace in objects") > >>>>> https://git.kernel.org/cgit/linux/kernel/git/vbabka/linux.git slub-stackdepot-v3r0 > >>>>> > >>>>> in testcase: boot > >>>>> > >>>>> on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G > >>>>> > >>>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): > >>>>> > >>>> > >>>> [+Cc Vlastimil and linux-mm] > >>> > >>> Thanks. > >>> lkp folks: it would be nice if I was CC'd automatically on this, it's a > >>> commit from my git tree and with by s-o-b :) > >>> > >>>> I _strongly_ suspect that this is because we don't initialize > >>>> stack_table[i] = NULL when we allocate it from memblock_alloc(). > >>> > >>> No, Mike (CC'd) suggested to drop the array init cycle, because > >>> memblock_alloc would zero the area anyway. > >> > >> Ah, you are right. My mistake. > >> > >>> There has to be a different > >>> reason. Wondering if dmesg contains the stack depot initialization message > >>> at all... > >> > >> I think I found the reason. > >> This is because of CONFIG_SLUB_DEBUG_ON. > >> It can enable debugging without passing boot parameter. > >> > >> if CONFIG_SLUB_DEBUG_ON=y && slub_debug is not passed, we do not call > >> stack_depot_want_early_init(), but the debugging flags are set. > >> > >> And we only call stack_depot_init() later in kmem_cache_create_usercopy(). > >> > >> so it crashed while creating boot cache. > > > > I tested this, and this was the reason. > > It crashed on CONFIG_SLUB_DEBUG_ON=y because stackdepot always assume > > that it was initialized in boot step, or failed > > (stack_depot_disable=true). > > > > But as it didn't even tried to initialize it, stack_table == NULL && > > stack_depot_disable == false. So accessing *(NULL + <hash value>) > > Thanks for finding the cause! > ;) > > Ideas? implementing something like kmem_cache_init_early() again? > > I think we could simply make CONFIG_SLUB_DEBUG_ON select/depend on > STACKDEPOT_ALWAYS_INIT? Oh, sounds better. If we make CONFIG_SLUB_DEBUG_ON select STACK_DEPOT_ALWAYS_INIT, that is simple solution. but stackdepot will be initialized on slub_debug=- too. But I think no one will set CONFIG_SLUB_DEBUG_ON=y if not debugging... I don't think making CONFIG_SLUB_DEBUG_ON depend on CONFIG_STACKDEPOT_ALWAYS_INIT is good solution. only KASAN selects it. -- Thank you, You are awesome! Hyeonggon :-)
next prev parent reply other threads:[~2022-03-12 11:04 UTC|newest] Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-03-09 2:15 [mm/slub] ae107fa919: BUG:unable_to_handle_page_fault_for_address kernel test robot 2022-03-09 2:15 ` kernel test robot 2022-03-11 14:54 ` Hyeonggon Yoo 2022-03-11 14:54 ` Hyeonggon Yoo 2022-03-11 15:36 ` Vlastimil Babka 2022-03-11 15:36 ` Vlastimil Babka 2022-03-11 16:46 ` Hyeonggon Yoo 2022-03-11 16:46 ` Hyeonggon Yoo 2022-03-12 1:10 ` Hyeonggon Yoo 2022-03-12 1:10 ` Hyeonggon Yoo 2022-03-12 9:21 ` Vlastimil Babka 2022-03-12 9:21 ` Vlastimil Babka 2022-03-12 11:04 ` Hyeonggon Yoo [this message] 2022-03-12 11:04 ` Hyeonggon Yoo 2022-03-12 12:06 ` Mike Rapoport 2022-03-12 12:06 ` Mike Rapoport 2022-03-12 13:08 ` Vlastimil Babka 2022-03-12 13:08 ` Vlastimil Babka 2022-03-14 16:20 ` Vlastimil Babka 2022-03-14 16:20 ` Vlastimil Babka
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=Yix+Pn9a9yKIiXC4@ip-172-31-19-208.ap-northeast-1.compute.internal \ --to=42.hyeyoo@gmail.com \ --cc=glittao@gmail.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lkp@intel.com \ --cc=lkp@lists.01.org \ --cc=oliver.sang@intel.com \ --cc=rppt@kernel.org \ --cc=vbabka@suse.cz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.