From: Dan Williams <dan.j.williams@intel.com> To: alexander.h.duyck@linux.intel.com Cc: "Brown, Len" <len.brown@intel.com>, bvanassche@acm.org, Linux-pm mailing list <linux-pm@vger.kernel.org>, Greg KH <gregkh@linuxfoundation.org>, linux-nvdimm <linux-nvdimm@lists.01.org>, jiangshanlai@gmail.com, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Pavel Machek <pavel@ucw.cz>, zwisler@kernel.org, Tejun Heo <tj@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, "Rafael J. Wysocki" <rafael@kernel.org> Subject: Re: [driver-core PATCH v6 9/9] libnvdimm: Schedule device registration on node local to the device Date: Tue, 27 Nov 2018 11:34:56 -0800 [thread overview] Message-ID: <CAPcyv4gWwsYOeXVNDoDy4v=SU6bRyBG1fgSu84y64jGApaJ91g@mail.gmail.com> (raw) In-Reply-To: <2031cf4705d76dd4d0f722a600a6a106cce2ba41.camel@linux.intel.com> On Tue, Nov 27, 2018 at 10:04 AM Alexander Duyck <alexander.h.duyck@linux.intel.com> wrote: > > On Mon, 2018-11-26 at 18:21 -0800, Dan Williams wrote: > > On Thu, Nov 8, 2018 at 10:07 AM Alexander Duyck > > <alexander.h.duyck@linux.intel.com> wrote: > > > > > > Force the device registration for nvdimm devices to be closer to the actual > > > device. This is achieved by using either the NUMA node ID of the region, or > > > of the parent. By doing this we can have everything above the region based > > > on the region, and everything below the region based on the nvdimm bus. > > > > > > By guaranteeing NUMA locality I see an improvement of as high as 25% for > > > per-node init of a system with 12TB of persistent memory. > > > > > > > It seems the speed-up is achieved with just patches 1, 2, and 9 from > > this series, correct? I wouldn't want to hold up that benefit while > > the driver-core bits are debated. > > Actually patch 6 ends up impacting things for persistent memory as > well. The problem is that all the async calls to add interfaces only do > anything if the driver is already loaded. So there are cases such as > the X86_PMEM_LEGACY_DEVICE case where the memory regions end up still > being serialized because the devices are added before the driver. Ok, but is the patch 6 change generally useful outside of the libnvdimm case? Yes, local hacks like MODULE_SOFTDEP are terrible for global problems, but what I'm trying to tease out if this change benefits other async probing subsystems outside of libnvdimm, SCSI perhaps? Bart can you chime in with the benefits you see so it's clear to Greg that the driver-core changes are a generic improvement? > > You can add: > > > > Reviewed-by: Dan Williams <dan.j.williams@intel.com> > > > > ...if the series needs to be kept together, but as far as I can see > > the workqueue changes enable 2 sub-topics of development and it might > > make sense for Tejun to take those first 2 and then Greg and I can > > base any follow-up topics on that stable baseline. > > I had originally put this out there for Tejun to apply, but him and > Greg had talked and Greg agreed to apply the set. If it works for you I > would prefer to just keep it together for now as I don't believe there > will be too many more revisions of this needed. > That works for me. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com> To: alexander.h.duyck@linux.intel.com Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Greg KH <gregkh@linuxfoundation.org>, linux-nvdimm <linux-nvdimm@lists.01.org>, Tejun Heo <tj@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Linux-pm mailing list <linux-pm@vger.kernel.org>, jiangshanlai@gmail.com, "Rafael J. Wysocki" <rafael@kernel.org>, "Brown, Len" <len.brown@intel.com>, Pavel Machek <pavel@ucw.cz>, zwisler@kernel.org, Dave Jiang <dave.jiang@intel.com>, bvanassche@acm.org Subject: Re: [driver-core PATCH v6 9/9] libnvdimm: Schedule device registration on node local to the device Date: Tue, 27 Nov 2018 11:34:56 -0800 [thread overview] Message-ID: <CAPcyv4gWwsYOeXVNDoDy4v=SU6bRyBG1fgSu84y64jGApaJ91g@mail.gmail.com> (raw) In-Reply-To: <2031cf4705d76dd4d0f722a600a6a106cce2ba41.camel@linux.intel.com> On Tue, Nov 27, 2018 at 10:04 AM Alexander Duyck <alexander.h.duyck@linux.intel.com> wrote: > > On Mon, 2018-11-26 at 18:21 -0800, Dan Williams wrote: > > On Thu, Nov 8, 2018 at 10:07 AM Alexander Duyck > > <alexander.h.duyck@linux.intel.com> wrote: > > > > > > Force the device registration for nvdimm devices to be closer to the actual > > > device. This is achieved by using either the NUMA node ID of the region, or > > > of the parent. By doing this we can have everything above the region based > > > on the region, and everything below the region based on the nvdimm bus. > > > > > > By guaranteeing NUMA locality I see an improvement of as high as 25% for > > > per-node init of a system with 12TB of persistent memory. > > > > > > > It seems the speed-up is achieved with just patches 1, 2, and 9 from > > this series, correct? I wouldn't want to hold up that benefit while > > the driver-core bits are debated. > > Actually patch 6 ends up impacting things for persistent memory as > well. The problem is that all the async calls to add interfaces only do > anything if the driver is already loaded. So there are cases such as > the X86_PMEM_LEGACY_DEVICE case where the memory regions end up still > being serialized because the devices are added before the driver. Ok, but is the patch 6 change generally useful outside of the libnvdimm case? Yes, local hacks like MODULE_SOFTDEP are terrible for global problems, but what I'm trying to tease out if this change benefits other async probing subsystems outside of libnvdimm, SCSI perhaps? Bart can you chime in with the benefits you see so it's clear to Greg that the driver-core changes are a generic improvement? > > You can add: > > > > Reviewed-by: Dan Williams <dan.j.williams@intel.com> > > > > ...if the series needs to be kept together, but as far as I can see > > the workqueue changes enable 2 sub-topics of development and it might > > make sense for Tejun to take those first 2 and then Greg and I can > > base any follow-up topics on that stable baseline. > > I had originally put this out there for Tejun to apply, but him and > Greg had talked and Greg agreed to apply the set. If it works for you I > would prefer to just keep it together for now as I don't believe there > will be too many more revisions of this needed. > That works for me.
next prev parent reply other threads:[~2018-11-27 19:35 UTC|newest] Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-11-08 18:06 [driver-core PATCH v6 0/9] Add NUMA aware async_schedule calls Alexander Duyck 2018-11-08 18:06 ` Alexander Duyck 2018-11-08 18:06 ` [driver-core PATCH v6 1/9] workqueue: Provide queue_work_node to queue work near a given NUMA node Alexander Duyck 2018-11-08 18:06 ` Alexander Duyck 2018-11-27 1:01 ` Dan Williams 2018-11-27 1:01 ` Dan Williams 2018-11-08 18:06 ` [driver-core PATCH v6 2/9] async: Add support for queueing on specific " Alexander Duyck 2018-11-08 18:06 ` Alexander Duyck 2018-11-08 23:36 ` Bart Van Assche 2018-11-08 23:36 ` Bart Van Assche 2018-11-08 23:36 ` Bart Van Assche 2018-11-11 19:32 ` Greg KH 2018-11-11 19:32 ` Greg KH 2018-11-11 19:32 ` Greg KH 2018-11-11 19:53 ` Dan Williams 2018-11-11 19:53 ` Dan Williams 2018-11-11 19:53 ` Dan Williams 2018-11-11 20:35 ` Greg KH 2018-11-11 20:35 ` Greg KH 2018-11-11 20:35 ` Greg KH 2018-11-11 22:17 ` Dan Williams 2018-11-11 22:17 ` Dan Williams 2018-11-11 22:17 ` Dan Williams 2018-11-11 23:27 ` Alexander Duyck 2018-11-11 23:27 ` Alexander Duyck 2018-11-11 23:27 ` Alexander Duyck 2018-11-11 19:59 ` Pavel Machek 2018-11-11 20:33 ` Greg KH 2018-11-11 20:33 ` Greg KH 2018-11-11 21:24 ` Bart Van Assche 2018-11-11 21:24 ` Bart Van Assche 2018-11-13 22:10 ` Pavel Machek 2018-11-27 1:10 ` Dan Williams 2018-11-27 1:10 ` Dan Williams 2018-11-27 1:10 ` Dan Williams 2018-11-08 18:06 ` [driver-core PATCH v6 3/9] device core: Consolidate locking and unlocking of parent and device Alexander Duyck 2018-11-08 18:06 ` Alexander Duyck 2018-11-08 18:06 ` Alexander Duyck 2018-11-08 22:43 ` jane.chu 2018-11-08 22:43 ` jane.chu-QHcLZuEGTsvQT0dZR+AlfA 2018-11-08 22:43 ` jane.chu 2018-11-08 22:48 ` Alexander Duyck 2018-11-08 22:48 ` Alexander Duyck 2018-11-27 1:44 ` Dan Williams 2018-11-27 1:44 ` Dan Williams 2018-11-08 18:07 ` [driver-core PATCH v6 4/9] driver core: Move async_synchronize_full call Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-27 2:11 ` Dan Williams 2018-11-27 2:11 ` Dan Williams 2018-11-27 17:38 ` Alexander Duyck 2018-11-27 17:38 ` Alexander Duyck 2018-11-27 20:35 ` Dan Williams 2018-11-27 20:35 ` Dan Williams 2018-11-27 21:36 ` Alexander Duyck 2018-11-27 21:36 ` Alexander Duyck 2018-11-27 22:26 ` Dan Williams 2018-11-27 22:26 ` Dan Williams 2018-11-27 22:26 ` Dan Williams 2018-11-08 18:07 ` [driver-core PATCH v6 5/9] driver core: Establish clear order of operations for deferred probe and remove Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-08 23:47 ` Bart Van Assche 2018-11-08 23:47 ` Bart Van Assche 2018-11-08 23:47 ` Bart Van Assche 2018-11-08 18:07 ` [driver-core PATCH v6 6/9] driver core: Probe devices asynchronously instead of the driver Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-08 23:59 ` Bart Van Assche 2018-11-08 23:59 ` Bart Van Assche 2018-11-27 2:48 ` Dan Williams 2018-11-27 2:48 ` Dan Williams 2018-11-27 2:48 ` Dan Williams 2018-11-27 17:57 ` Alexander Duyck 2018-11-27 17:57 ` Alexander Duyck 2018-11-27 18:32 ` Dan Williams 2018-11-27 18:32 ` Dan Williams 2018-11-08 18:07 ` [driver-core PATCH v6 7/9] driver core: Attach devices on CPU local to device node Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-27 4:50 ` Dan Williams 2018-11-27 4:50 ` Dan Williams 2018-11-27 4:50 ` Dan Williams 2018-11-08 18:07 ` [driver-core PATCH v6 8/9] PM core: Use new async_schedule_dev command Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-27 4:52 ` Dan Williams 2018-11-27 4:52 ` Dan Williams 2018-11-27 4:52 ` Dan Williams 2018-11-08 18:07 ` [driver-core PATCH v6 9/9] libnvdimm: Schedule device registration on node local to the device Alexander Duyck 2018-11-08 18:07 ` Alexander Duyck 2018-11-27 2:21 ` Dan Williams 2018-11-27 2:21 ` Dan Williams 2018-11-27 2:21 ` Dan Williams 2018-11-27 18:04 ` Alexander Duyck 2018-11-27 19:34 ` Dan Williams [this message] 2018-11-27 19:34 ` Dan Williams 2018-11-27 20:33 ` Bart Van Assche 2018-11-27 20:33 ` Bart Van Assche 2018-11-27 20:33 ` Bart Van Assche 2018-11-27 20:50 ` Dan Williams 2018-11-27 20:50 ` Dan Williams 2018-11-27 20:50 ` Dan Williams 2018-11-27 21:22 ` Bart Van Assche 2018-11-27 21:22 ` Bart Van Assche 2018-11-27 21:22 ` Bart Van Assche 2018-11-27 22:34 ` Dan Williams 2018-11-27 22:34 ` Dan Williams 2018-11-27 22:34 ` Dan Williams
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAPcyv4gWwsYOeXVNDoDy4v=SU6bRyBG1fgSu84y64jGApaJ91g@mail.gmail.com' \ --to=dan.j.williams@intel.com \ --cc=akpm@linux-foundation.org \ --cc=alexander.h.duyck@linux.intel.com \ --cc=bvanassche@acm.org \ --cc=gregkh@linuxfoundation.org \ --cc=jiangshanlai@gmail.com \ --cc=len.brown@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvdimm@lists.01.org \ --cc=linux-pm@vger.kernel.org \ --cc=pavel@ucw.cz \ --cc=rafael@kernel.org \ --cc=tj@kernel.org \ --cc=zwisler@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.