From: Alexander Duyck <alexander.h.duyck@linux.intel.com> To: Dan Williams <dan.j.williams@intel.com> Cc: "Pasha Tatashin" <pavel.tatashin@microsoft.com>, "Michal Hocko" <mhocko@suse.com>, linux-nvdimm <linux-nvdimm@lists.01.org>, "Dave Hansen" <dave.hansen@intel.com>, "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>, "Linux MM" <linux-mm@kvack.org>, "Jérôme Glisse" <jglisse@redhat.com>, rppt@linux.vnet.ibm.com, "Andrew Morton" <akpm@linux-foundation.org>, "Ingo Molnar" <mingo@kernel.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: Re: [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Date: Mon, 8 Oct 2018 14:38:05 -0700 [thread overview] Message-ID: <379e1d22-4194-6744-9e80-897b6ba126e9@linux.intel.com> (raw) In-Reply-To: <CAPcyv4jX5WYmMYzGCBrnaqT7tqHGSVPwm7Dpi-XpuM9ns84+0w@mail.gmail.com> On 10/8/2018 2:01 PM, Dan Williams wrote: > On Tue, Sep 25, 2018 at 1:29 PM Alexander Duyck > <alexander.h.duyck@linux.intel.com> wrote: >> >> The ZONE_DEVICE pages were being initialized in two locations. One was with >> the memory_hotplug lock held and another was outside of that lock. The >> problem with this is that it was nearly doubling the memory initialization >> time. Instead of doing this twice, once while holding a global lock and >> once without, I am opting to defer the initialization to the one outside of >> the lock. This allows us to avoid serializing the overhead for memory init >> and we can instead focus on per-node init times. >> >> One issue I encountered is that devm_memremap_pages and >> hmm_devmmem_pages_create were initializing only the pgmap field the same >> way. One wasn't initializing hmm_data, and the other was initializing it to >> a poison value. Since this is something that is exposed to the driver in >> the case of hmm I am opting for a third option and just initializing >> hmm_data to 0 since this is going to be exposed to unknown third party >> drivers. >> >> Reviewed-by: Pavel Tatashin <pavel.tatashin@microsoft.com> >> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com> >> --- >> >> v4: Moved moved memmap_init_zone_device to below memmmap_init_zone to avoid >> merge conflicts with other changes in the kernel. >> v5: No change > > This patch appears to cause a regression in the "create.sh" unit test > in the ndctl test suite. So all you had to do is run the create.sh script to see the issue? I just want to confirm there isn't any additional information needed before I try chasing this down. > I tried to reproduce on -next with: > > 2302f5ee215e mm: defer ZONE_DEVICE page initialization to the point > where we init pgmap > > ...but -next does not even boot for me at that commit. What version of -next? There are a couple of patches probably needed depending on which version you are trying to boot. > Here is a warning signature that proceeds a hang with this patch > applied against v4.19-rc6: > > percpu ref (blk_queue_usage_counter_release) <= 0 (-1530626) after > switching to atomic > WARNING: CPU: 24 PID: 7346 at lib/percpu-refcount.c:155 > percpu_ref_switch_to_atomic_rcu+0x1f7/0x200 > CPU: 24 PID: 7346 Comm: modprobe Tainted: G OE 4.19.0-rc6+ #2458 > [..] > RIP: 0010:percpu_ref_switch_to_atomic_rcu+0x1f7/0x200 > [..] > Call Trace: > <IRQ> > ? percpu_ref_reinit+0x140/0x140 > rcu_process_callbacks+0x273/0x880 > __do_softirq+0xd2/0x428 > irq_exit+0xf6/0x100 > smp_apic_timer_interrupt+0xa2/0x220 > apic_timer_interrupt+0xf/0x20 > </IRQ> > RIP: 0010:lock_acquire+0xb8/0x1a0 > [..] > ? __put_page+0x55/0x150 > ? __put_page+0x55/0x150 > __put_page+0x83/0x150 > ? __put_page+0x55/0x150 > devm_memremap_pages_release+0x194/0x250 > release_nodes+0x17c/0x2c0 > device_release_driver_internal+0x1a2/0x250 > driver_detach+0x3a/0x70 > bus_remove_driver+0x58/0xd0 > __x64_sys_delete_module+0x13f/0x200 > ? trace_hardirqs_off_thunk+0x1a/0x1c > do_syscall_64+0x60/0x210 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > So it looks like we are tearing down memory when this is triggered. Do we know if this is at the end of the test or if this is running in parallel with anything? _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Alexander Duyck <alexander.h.duyck@linux.intel.com> To: Dan Williams <dan.j.williams@intel.com> Cc: "Linux MM" <linux-mm@kvack.org>, "Andrew Morton" <akpm@linux-foundation.org>, "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>, linux-nvdimm <linux-nvdimm@lists.01.org>, "Pasha Tatashin" <pavel.tatashin@microsoft.com>, "Michal Hocko" <mhocko@suse.com>, "Dave Jiang" <dave.jiang@intel.com>, "Dave Hansen" <dave.hansen@intel.com>, "Jérôme Glisse" <jglisse@redhat.com>, rppt@linux.vnet.ibm.com, "Logan Gunthorpe" <logang@deltatee.com>, "Ingo Molnar" <mingo@kernel.org>, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: Re: [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Date: Mon, 8 Oct 2018 14:38:05 -0700 [thread overview] Message-ID: <379e1d22-4194-6744-9e80-897b6ba126e9@linux.intel.com> (raw) In-Reply-To: <CAPcyv4jX5WYmMYzGCBrnaqT7tqHGSVPwm7Dpi-XpuM9ns84+0w@mail.gmail.com> On 10/8/2018 2:01 PM, Dan Williams wrote: > On Tue, Sep 25, 2018 at 1:29 PM Alexander Duyck > <alexander.h.duyck@linux.intel.com> wrote: >> >> The ZONE_DEVICE pages were being initialized in two locations. One was with >> the memory_hotplug lock held and another was outside of that lock. The >> problem with this is that it was nearly doubling the memory initialization >> time. Instead of doing this twice, once while holding a global lock and >> once without, I am opting to defer the initialization to the one outside of >> the lock. This allows us to avoid serializing the overhead for memory init >> and we can instead focus on per-node init times. >> >> One issue I encountered is that devm_memremap_pages and >> hmm_devmmem_pages_create were initializing only the pgmap field the same >> way. One wasn't initializing hmm_data, and the other was initializing it to >> a poison value. Since this is something that is exposed to the driver in >> the case of hmm I am opting for a third option and just initializing >> hmm_data to 0 since this is going to be exposed to unknown third party >> drivers. >> >> Reviewed-by: Pavel Tatashin <pavel.tatashin@microsoft.com> >> Signed-off-by: Alexander Duyck <alexander.h.duyck@linux.intel.com> >> --- >> >> v4: Moved moved memmap_init_zone_device to below memmmap_init_zone to avoid >> merge conflicts with other changes in the kernel. >> v5: No change > > This patch appears to cause a regression in the "create.sh" unit test > in the ndctl test suite. So all you had to do is run the create.sh script to see the issue? I just want to confirm there isn't any additional information needed before I try chasing this down. > I tried to reproduce on -next with: > > 2302f5ee215e mm: defer ZONE_DEVICE page initialization to the point > where we init pgmap > > ...but -next does not even boot for me at that commit. What version of -next? There are a couple of patches probably needed depending on which version you are trying to boot. > Here is a warning signature that proceeds a hang with this patch > applied against v4.19-rc6: > > percpu ref (blk_queue_usage_counter_release) <= 0 (-1530626) after > switching to atomic > WARNING: CPU: 24 PID: 7346 at lib/percpu-refcount.c:155 > percpu_ref_switch_to_atomic_rcu+0x1f7/0x200 > CPU: 24 PID: 7346 Comm: modprobe Tainted: G OE 4.19.0-rc6+ #2458 > [..] > RIP: 0010:percpu_ref_switch_to_atomic_rcu+0x1f7/0x200 > [..] > Call Trace: > <IRQ> > ? percpu_ref_reinit+0x140/0x140 > rcu_process_callbacks+0x273/0x880 > __do_softirq+0xd2/0x428 > irq_exit+0xf6/0x100 > smp_apic_timer_interrupt+0xa2/0x220 > apic_timer_interrupt+0xf/0x20 > </IRQ> > RIP: 0010:lock_acquire+0xb8/0x1a0 > [..] > ? __put_page+0x55/0x150 > ? __put_page+0x55/0x150 > __put_page+0x83/0x150 > ? __put_page+0x55/0x150 > devm_memremap_pages_release+0x194/0x250 > release_nodes+0x17c/0x2c0 > device_release_driver_internal+0x1a2/0x250 > driver_detach+0x3a/0x70 > bus_remove_driver+0x58/0xd0 > __x64_sys_delete_module+0x13f/0x200 > ? trace_hardirqs_off_thunk+0x1a/0x1c > do_syscall_64+0x60/0x210 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > So it looks like we are tearing down memory when this is triggered. Do we know if this is at the end of the test or if this is running in parallel with anything?
next prev parent reply other threads:[~2018-10-08 21:38 UTC|newest] Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-09-25 20:18 [PATCH v5 0/4] Address issues slowing persistent memory initialization Alexander Duyck 2018-09-25 20:18 ` Alexander Duyck 2018-09-25 20:19 ` [PATCH v5 1/4] mm: Remove now defunct NO_BOOTMEM from depends list for deferred init Alexander Duyck 2018-09-25 20:19 ` Alexander Duyck 2018-09-25 21:05 ` Mike Rapoport 2018-09-25 21:05 ` Mike Rapoport 2018-09-25 20:20 ` [PATCH v5 2/4] mm: Provide kernel parameter to allow disabling page init poisoning Alexander Duyck 2018-09-25 20:20 ` Alexander Duyck 2018-09-25 20:26 ` Dave Hansen 2018-09-25 20:26 ` Dave Hansen 2018-09-25 20:38 ` Alexander Duyck 2018-09-25 20:38 ` Alexander Duyck 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:14 ` Dave Hansen 2018-09-25 22:27 ` Alexander Duyck 2018-09-25 22:27 ` Alexander Duyck 2018-09-25 22:27 ` Alexander Duyck 2018-09-26 7:38 ` Michal Hocko 2018-09-26 7:38 ` Michal Hocko 2018-09-26 15:24 ` Alexander Duyck 2018-09-26 15:39 ` Michal Hocko 2018-09-26 15:39 ` Michal Hocko 2018-09-26 15:41 ` Dave Hansen 2018-09-26 15:41 ` Dave Hansen 2018-09-26 16:18 ` Alexander Duyck 2018-09-26 15:36 ` Dave Hansen 2018-09-26 22:36 ` Andrew Morton 2018-09-26 22:36 ` Andrew Morton 2018-09-25 20:20 ` [PATCH v5 3/4] mm: Create non-atomic version of SetPageReserved for init use Alexander Duyck 2018-09-25 20:20 ` Alexander Duyck 2018-09-25 20:21 ` [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Alexander Duyck 2018-09-25 20:21 ` Alexander Duyck 2018-09-26 7:55 ` Michal Hocko 2018-09-26 18:25 ` Alexander Duyck 2018-09-26 18:25 ` Alexander Duyck 2018-09-26 18:52 ` Dan Williams 2018-09-26 18:52 ` Dan Williams 2018-09-27 11:20 ` Michal Hocko 2018-09-27 11:20 ` Michal Hocko 2018-09-27 11:09 ` Michal Hocko 2018-09-27 11:09 ` Michal Hocko 2018-09-27 12:25 ` Oscar Salvador 2018-09-27 13:13 ` Michal Hocko 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 14:50 ` Oscar Salvador 2018-09-27 15:41 ` David Hildenbrand 2018-09-27 15:41 ` David Hildenbrand 2018-09-28 8:12 ` Oscar Salvador 2018-09-28 8:12 ` Oscar Salvador 2018-09-28 8:44 ` Oscar Salvador 2018-09-28 8:44 ` Oscar Salvador 2018-09-28 15:50 ` Dan Williams 2018-09-28 15:50 ` Dan Williams 2018-09-27 12:32 ` Oscar Salvador 2018-10-08 21:01 ` Dan Williams 2018-10-08 21:01 ` Dan Williams 2018-10-08 21:38 ` Alexander Duyck [this message] 2018-10-08 21:38 ` Alexander Duyck 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:00 ` Dan Williams 2018-10-08 22:07 ` Alexander Duyck 2018-10-08 22:07 ` Alexander Duyck 2018-10-08 22:36 ` Alexander Duyck 2018-10-08 22:36 ` Alexander Duyck 2018-10-08 22:59 ` Dan Williams 2018-10-08 23:34 ` [mm PATCH] memremap: Fix reference count for pgmap in devm_memremap_pages Alexander Duyck 2018-10-08 23:34 ` Alexander Duyck 2018-10-09 0:20 ` Dan Williams 2018-10-09 0:20 ` Dan Williams 2018-10-09 17:00 ` [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap Yi Zhang 2018-10-09 17:00 ` Yi Zhang 2018-10-09 18:04 ` Dan Williams 2018-10-09 18:04 ` Dan Williams 2018-10-09 20:26 ` Alexander Duyck 2018-10-09 20:26 ` Alexander Duyck 2018-10-09 21:19 ` Dan Williams 2018-10-09 21:19 ` Dan Williams 2018-10-10 12:52 ` Yi Zhang 2018-10-10 12:52 ` Yi Zhang 2018-10-10 15:27 ` Alexander Duyck 2018-10-10 15:27 ` Alexander Duyck 2018-10-11 8:17 ` Yi Zhang 2018-10-11 8:17 ` Yi Zhang 2018-10-10 9:58 ` Michal Hocko 2018-10-10 16:39 ` Alexander Duyck 2018-10-10 16:39 ` Alexander Duyck 2018-10-10 17:24 ` Michal Hocko 2018-10-10 17:24 ` Michal Hocko 2018-10-10 17:39 ` Alexander Duyck 2018-10-10 17:39 ` Alexander Duyck 2018-10-10 17:53 ` Michal Hocko 2018-10-10 17:53 ` Michal Hocko 2018-10-10 18:13 ` Alexander Duyck 2018-10-10 18:13 ` Alexander Duyck 2018-10-10 18:52 ` Michal Hocko 2018-10-10 18:52 ` Michal Hocko 2018-10-11 8:55 ` Michal Hocko 2018-10-11 8:55 ` Michal Hocko 2018-10-11 17:38 ` Alexander Duyck 2018-10-11 18:22 ` Dan Williams 2018-10-11 18:22 ` Dan Williams 2018-10-17 7:52 ` Michal Hocko 2018-10-17 7:52 ` Michal Hocko 2018-10-17 15:02 ` Alexander Duyck 2018-10-17 15:02 ` Alexander Duyck 2018-10-29 14:12 ` Michal Hocko 2018-10-29 14:12 ` Michal Hocko 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 15:59 ` Alexander Duyck 2018-10-29 16:35 ` Michal Hocko 2018-10-29 16:35 ` Michal Hocko 2018-10-29 17:01 ` Alexander Duyck 2018-10-29 17:24 ` Michal Hocko 2018-10-29 17:24 ` Michal Hocko 2018-10-29 17:34 ` Dan Williams 2018-10-29 17:34 ` Dan Williams 2018-10-29 17:45 ` Michal Hocko 2018-10-29 17:45 ` Michal Hocko 2018-10-29 17:42 ` Alexander Duyck 2018-10-29 17:42 ` Alexander Duyck 2018-10-29 18:18 ` Michal Hocko 2018-10-29 18:18 ` Michal Hocko 2018-10-29 19:59 ` Alexander Duyck 2018-10-29 19:59 ` Alexander Duyck 2018-10-30 6:29 ` Michal Hocko 2018-10-30 6:29 ` Michal Hocko 2018-10-30 6:55 ` Dan Williams 2018-10-30 8:17 ` Michal Hocko 2018-10-30 8:17 ` Michal Hocko 2018-10-30 15:57 ` Dan Williams 2018-10-30 8:05 ` Oscar Salvador 2018-10-29 15:49 ` Dan Williams 2018-10-29 15:49 ` Dan Williams 2018-10-29 15:56 ` Michal Hocko 2018-10-10 18:18 ` Dan Williams 2018-10-10 18:18 ` Dan Williams 2018-10-11 8:39 ` Yi Zhang 2018-10-11 8:39 ` Yi Zhang 2018-10-11 15:38 ` Alexander Duyck 2018-10-11 15:38 ` Alexander Duyck
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=379e1d22-4194-6744-9e80-897b6ba126e9@linux.intel.com \ --to=alexander.h.duyck@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=dan.j.williams@intel.com \ --cc=dave.hansen@intel.com \ --cc=jglisse@redhat.com \ --cc=kirill.shutemov@linux.intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-nvdimm@lists.01.org \ --cc=mhocko@suse.com \ --cc=mingo@kernel.org \ --cc=pavel.tatashin@microsoft.com \ --cc=rppt@linux.vnet.ibm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.