From: Dan Williams <dan.j.williams@intel.com> To: Mike Snitzer <snitzer@redhat.com> Cc: "axboe@kernel.dk" <axboe@kernel.dk>, "dm-devel@redhat.com" <dm-devel@redhat.com>, "linux-nvdimm@lists.01.org" <linux-nvdimm@ml01.01.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-raid@vger.kernel.org" <linux-raid@vger.kernel.org>, "viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>, "agk@redhat.com" <agk@redhat.com> Subject: Re: [PATCH 0/6] Support DAX for device-mapper dm-linear devices Date: Tue, 14 Jun 2016 19:07:53 -0700 [thread overview] Message-ID: <CAPcyv4g6hRciQHA=QMf3BUKxHOpyZCUSX1E9Huc8U2GxqHK4vA@mail.gmail.com> (raw) In-Reply-To: <20160615014658.GA5443@redhat.com> On Tue, Jun 14, 2016 at 6:46 PM, Mike Snitzer <snitzer@redhat.com> wrote: > On Tue, Jun 14 2016 at 4:19pm -0400, > Jeff Moyer <jmoyer@redhat.com> wrote: > >> Mike Snitzer <snitzer@redhat.com> writes: >> >> > On Tue, Jun 14 2016 at 9:50am -0400, >> > Jeff Moyer <jmoyer@redhat.com> wrote: >> > >> >> "Kani, Toshimitsu" <toshi.kani@hpe.com> writes: >> >> >> >> >> I had dm-linear and md-raid0 support on my list of things to look at, >> >> >> did you have raid0 in your plans? >> >> > >> >> > Yes, I hope to extend further and raid0 is a good candidate. >> >> >> >> dm-flakey would allow more xfstests test cases to run. I'd say that's >> >> more important than linear or raid0. ;-) >> > >> > Regardless of which target(s) grow DAX support the most pressing initial >> > concern is getting the DM device stacking correct. And verifying that >> > IO that cross pmem device boundaries are being properly split by DM >> > core (via drivers/md/dm.c:__split_and_process_non_flush()'s call to >> > max_io_len). >> >> That was a tongue-in-cheek comment. You're reading way too much into >> it. >> >> >> Also, the next step in this work is to then decide how to determine on >> >> what numa node an LBA resides. We had discussed this at a prior >> >> plumbers conference, and I think the consensus was to use xattrs. >> >> Toshi, do you also plan to do that work? >> > >> > How does the associated NUMA node relate to this? Does the >> > DM requests_queue need to be setup to only allocate from the NUMA node >> > the pmem device is attached to? I recently added support for this to >> > DM. But there will likely be some code need to propagate the NUMA node >> > id accordingly. >> >> I assume you mean allocate memory (the volatile kind). That should work >> the same between pmem and regular block devices, no? > > This is the commit I made to train DM to be numa node aware: > 115485e83f497fdf9b4 ("dm: add 'dm_numa_node' module parameter") Hmm, but this is global for all DM device instances. > As is the DM code is focused on memory allocations. But I think blk-mq > may use the NUMA node for via tag_set->numa_node. But that is moot > given pmem is bio-based right? Right. > > Steps could be taken to make all threads DM creates for a a given device > get pinned to the specified NUMA node too. I think it would be useful if a DM instance inherited the numa node from the component devices by default (assuming they're all from the same node). A "dev_to_node(disk_to_dev(disk))" conversion works for pmem devices. As far as I understand, Jeff wants to go further and have a linear span across component devices from different nodes with an interface to do an LBA-to-numa-node conversion. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Dan Williams <dan.j.williams@intel.com> To: Mike Snitzer <snitzer@redhat.com> Cc: Jeff Moyer <jmoyer@redhat.com>, "Kani, Toshimitsu" <toshi.kani@hpe.com>, "axboe@kernel.dk" <axboe@kernel.dk>, "linux-nvdimm@lists.01.org" <linux-nvdimm@ml01.01.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "linux-raid@vger.kernel.org" <linux-raid@vger.kernel.org>, "dm-devel@redhat.com" <dm-devel@redhat.com>, "viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>, "ross.zwisler@linux.intel.com" <ross.zwisler@linux.intel.com>, "agk@redhat.com" <agk@redhat.com> Subject: Re: [PATCH 0/6] Support DAX for device-mapper dm-linear devices Date: Tue, 14 Jun 2016 19:07:53 -0700 [thread overview] Message-ID: <CAPcyv4g6hRciQHA=QMf3BUKxHOpyZCUSX1E9Huc8U2GxqHK4vA@mail.gmail.com> (raw) In-Reply-To: <20160615014658.GA5443@redhat.com> On Tue, Jun 14, 2016 at 6:46 PM, Mike Snitzer <snitzer@redhat.com> wrote: > On Tue, Jun 14 2016 at 4:19pm -0400, > Jeff Moyer <jmoyer@redhat.com> wrote: > >> Mike Snitzer <snitzer@redhat.com> writes: >> >> > On Tue, Jun 14 2016 at 9:50am -0400, >> > Jeff Moyer <jmoyer@redhat.com> wrote: >> > >> >> "Kani, Toshimitsu" <toshi.kani@hpe.com> writes: >> >> >> >> >> I had dm-linear and md-raid0 support on my list of things to look at, >> >> >> did you have raid0 in your plans? >> >> > >> >> > Yes, I hope to extend further and raid0 is a good candidate. >> >> >> >> dm-flakey would allow more xfstests test cases to run. I'd say that's >> >> more important than linear or raid0. ;-) >> > >> > Regardless of which target(s) grow DAX support the most pressing initial >> > concern is getting the DM device stacking correct. And verifying that >> > IO that cross pmem device boundaries are being properly split by DM >> > core (via drivers/md/dm.c:__split_and_process_non_flush()'s call to >> > max_io_len). >> >> That was a tongue-in-cheek comment. You're reading way too much into >> it. >> >> >> Also, the next step in this work is to then decide how to determine on >> >> what numa node an LBA resides. We had discussed this at a prior >> >> plumbers conference, and I think the consensus was to use xattrs. >> >> Toshi, do you also plan to do that work? >> > >> > How does the associated NUMA node relate to this? Does the >> > DM requests_queue need to be setup to only allocate from the NUMA node >> > the pmem device is attached to? I recently added support for this to >> > DM. But there will likely be some code need to propagate the NUMA node >> > id accordingly. >> >> I assume you mean allocate memory (the volatile kind). That should work >> the same between pmem and regular block devices, no? > > This is the commit I made to train DM to be numa node aware: > 115485e83f497fdf9b4 ("dm: add 'dm_numa_node' module parameter") Hmm, but this is global for all DM device instances. > As is the DM code is focused on memory allocations. But I think blk-mq > may use the NUMA node for via tag_set->numa_node. But that is moot > given pmem is bio-based right? Right. > > Steps could be taken to make all threads DM creates for a a given device > get pinned to the specified NUMA node too. I think it would be useful if a DM instance inherited the numa node from the component devices by default (assuming they're all from the same node). A "dev_to_node(disk_to_dev(disk))" conversion works for pmem devices. As far as I understand, Jeff wants to go further and have a linear span across component devices from different nodes with an interface to do an LBA-to-numa-node conversion.
next prev parent reply other threads:[~2016-06-15 2:07 UTC|newest] Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-06-13 22:21 [PATCH 0/6] Support DAX for device-mapper dm-linear devices Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 1/6] genhd: Add GENHD_FL_DAX to gendisk flags Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 2/6] block: Check GENHD_FL_DAX for DAX capability Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 3/6] dm: Add dm_blk_direct_access() for mapped device Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 4/6] dm-linear: Add linear_direct_access() Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 5/6] dm, dm-linear: Add dax_supported to dm_target Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` [PATCH 6/6] dm: Enable DAX support for mapper device Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:21 ` Toshi Kani 2016-06-13 22:57 ` [PATCH 0/6] Support DAX for device-mapper dm-linear devices Mike Snitzer [not found] ` <20160613225756.GA18417-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-20 18:00 ` Mike Snitzer 2016-06-20 18:00 ` Mike Snitzer [not found] ` <20160620180043.GA21261-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-20 18:31 ` Kani, Toshimitsu 2016-06-20 18:31 ` Kani, Toshimitsu [not found] ` <1466446861.3504.243.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-20 19:40 ` Mike Snitzer 2016-06-20 19:40 ` Mike Snitzer [not found] ` <20160620194026.GA21657-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-20 19:52 ` Mike Snitzer 2016-06-20 19:52 ` Mike Snitzer [not found] ` <20160620195217.GB21657-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-20 20:11 ` Kani, Toshimitsu 2016-06-20 20:11 ` Kani, Toshimitsu [not found] ` <1466452883.3504.244.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-20 21:28 ` Kani, Toshimitsu 2016-06-20 21:28 ` Kani, Toshimitsu [not found] ` <1466457467.3504.249.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-20 22:22 ` Mike Snitzer 2016-06-20 22:22 ` Mike Snitzer [not found] ` <20160620222236.GA22461-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-21 13:41 ` Mike Snitzer 2016-06-21 13:41 ` Mike Snitzer [not found] ` <20160621134147.GA26392-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-21 15:44 ` Kani, Toshimitsu 2016-06-21 15:44 ` Kani, Toshimitsu [not found] ` <1466523280.3504.262.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-21 15:50 ` Kani, Toshimitsu 2016-06-21 15:50 ` Kani, Toshimitsu 2016-06-21 16:25 ` Dan Williams 2016-06-21 16:25 ` Dan Williams [not found] ` <CAPcyv4gFREc94ANuFD_Lyddx3iqRTN2UDebgeJe3LqPL8xrVzg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-06-21 16:35 ` Kani, Toshimitsu 2016-06-21 16:35 ` Kani, Toshimitsu [not found] ` <1466526342.3504.270.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-21 16:45 ` Dan Williams 2016-06-21 16:45 ` Dan Williams [not found] ` <CAPcyv4ht8B7dHe1ckv5d=bOrRzCy3=ZDVSTD0rRsak_LYD8r8g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2016-06-21 16:56 ` Kani, Toshimitsu 2016-06-21 16:56 ` Kani, Toshimitsu 2016-06-21 18:17 ` Mike Snitzer 2016-06-21 18:17 ` Mike Snitzer 2016-06-22 17:44 ` Kani, Toshimitsu 2016-06-22 17:44 ` Kani, Toshimitsu [not found] ` <1466616868.3504.320.camel-ZPxbGqLxI0U@public.gmane.org> 2016-06-22 19:15 ` Dan Williams 2016-06-22 19:15 ` Dan Williams 2016-06-22 20:16 ` Kani, Toshimitsu 2016-06-22 22:38 ` Mike Snitzer 2016-06-22 22:38 ` Mike Snitzer [not found] ` <20160622223842.GA34512-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2016-06-22 22:59 ` Kani, Toshimitsu 2016-06-22 22:59 ` Kani, Toshimitsu 2016-06-13 23:18 ` Dan Williams 2016-06-13 23:18 ` Dan Williams 2016-06-13 23:18 ` Dan Williams 2016-06-13 23:59 ` Kani, Toshimitsu 2016-06-13 23:59 ` Kani, Toshimitsu 2016-06-13 23:59 ` Kani, Toshimitsu 2016-06-14 0:02 ` Dan Williams 2016-06-14 0:02 ` Dan Williams 2016-06-14 0:02 ` Dan Williams 2016-06-14 7:30 ` Dan Williams 2016-06-14 7:30 ` Dan Williams 2016-06-14 7:30 ` Dan Williams 2016-06-14 13:50 ` Jeff Moyer 2016-06-14 13:50 ` Jeff Moyer 2016-06-14 13:50 ` Jeff Moyer 2016-06-14 15:41 ` Mike Snitzer 2016-06-14 15:41 ` Mike Snitzer 2016-06-14 15:41 ` Mike Snitzer 2016-06-14 18:00 ` Kani, Toshimitsu 2016-06-14 20:19 ` Jeff Moyer 2016-06-14 20:19 ` Jeff Moyer 2016-06-14 20:19 ` Jeff Moyer 2016-06-15 1:46 ` Mike Snitzer 2016-06-15 1:46 ` Mike Snitzer 2016-06-15 2:07 ` Dan Williams [this message] 2016-06-15 2:07 ` Dan Williams 2016-06-15 2:35 ` Mike Snitzer 2016-06-15 2:35 ` Mike Snitzer 2016-06-14 15:53 ` Kani, Toshimitsu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAPcyv4g6hRciQHA=QMf3BUKxHOpyZCUSX1E9Huc8U2GxqHK4vA@mail.gmail.com' \ --to=dan.j.williams@intel.com \ --cc=agk@redhat.com \ --cc=axboe@kernel.dk \ --cc=dm-devel@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvdimm@ml01.01.org \ --cc=linux-raid@vger.kernel.org \ --cc=snitzer@redhat.com \ --cc=viro@zeniv.linux.org.uk \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.