From: Dave Hansen <dave.hansen@intel.com>
To: Pavel Tatashin <pasha.tatashin@soleen.com>,
jmorris@namei.org, sashal@kernel.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
linux-nvdimm@lists.01.org, akpm@linux-foundation.org,
mhocko@suse.com, dave.hansen@linux.intel.com,
dan.j.williams@intel.com, keith.busch@intel.com,
vishal.l.verma@intel.com, dave.jiang@intel.com,
zwisler@kernel.org, thomas.lendacky@amd.com,
ying.huang@intel.com, fengguang.wu@intel.com, bp@suse.de,
bhelgaas@google.com, baiyaowei@cmss.chinamobile.com,
tiwai@suse.de, jglisse@redhat.com, david@redhat.com
Subject: Re: [v3 2/2] device-dax: "Hotremove" persistent memory that is used like normal RAM
Date: Thu, 25 Apr 2019 12:01:33 -0700 [thread overview]
Message-ID: <77c286e3-8708-6e64-94a1-fb44b6bbff3f@intel.com> (raw)
In-Reply-To: <20190425175440.9354-3-pasha.tatashin@soleen.com>
Hi Pavel,
Thanks for doing this! I knew we'd have to get to it eventually, but
sounds like you needed it sooner rather than later.
...
> static inline struct dev_dax *to_dev_dax(struct device *dev)
> diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> index 4c0131857133..6f1640462df9 100644
> --- a/drivers/dax/kmem.c
> +++ b/drivers/dax/kmem.c
> @@ -71,21 +71,107 @@ int dev_dax_kmem_probe(struct device *dev)
> kfree(new_res);
> return rc;
> }
> + dev_dax->dax_kmem_res = new_res;
>
> return 0;
> }
>
> +#ifdef CONFIG_MEMORY_HOTREMOVE
Instead of this #ifdef, is there any downside to doing
if (!IS_ENABLED(CONFIG_MEMORY_HOTREMOVE)) {
/*
* Without hotremove, purposely leak ...
*/
return 0;
}
> +/*
> + * Check that device-dax's memory_blocks are offline. If a memory_block is not
> + * offline a warning is printed and an error is returned. dax hotremove can
> + * succeed only when every memory_block is offlined beforehand.
> + */
I'd much rather see comments inline with the code than all piled at the
top of a function like this.
One thing that would be helpful, though, is a reminder about needing the
device hotplug lock.
> +static int
> +check_memblock_offlined_cb(struct memory_block *mem, void *arg)
> +{
> + struct device *mem_dev = &mem->dev;
> + bool is_offline;
> +
> + device_lock(mem_dev);
> + is_offline = mem_dev->offline;
> + device_unlock(mem_dev);
> +
> + if (!is_offline) {
> + struct device *dev = (struct device *)arg;
The two devices confused me for a bit here. Seems worth a comment to
remind the reader what this device _is_ versus 'mem_dev'.
> + unsigned long spfn = section_nr_to_pfn(mem->start_section_nr);
> + unsigned long epfn = section_nr_to_pfn(mem->end_section_nr);
> + phys_addr_t spa = spfn << PAGE_SHIFT;
> + phys_addr_t epa = epfn << PAGE_SHIFT;
> +
> + dev_warn(dev, "memory block [%pa-%pa] is not offline\n",
> + &spa, &epa);
I thought we had a magic resource printk %something. Could we just
print one of the device resources here to save all the section/pfn/paddr
calculations?
Also, should we consider a slightly scarier message? This path has a
permanent, user-visible effect (we can never try to unbind again).
> + return -EBUSY;
> + }
> +
> + return 0;
> +}
Even though they're static, I'd prefer that we not create two versions
of check_memblock_offlined_cb() in the kernel. Can we give this a
better, non-conflicting name?
> +static int dev_dax_kmem_remove(struct device *dev)
> +{
> + struct dev_dax *dev_dax = to_dev_dax(dev);
> + struct resource *res = dev_dax->dax_kmem_res;
> + resource_size_t kmem_start;
> + resource_size_t kmem_size;
> + unsigned long start_pfn;
> + unsigned long end_pfn;
> + int rc;
> +
> + /*
> + * dax kmem resource does not exist, means memory was never hotplugged.
> + * So, nothing to do here.
> + */
> + if (!res)
> + return 0;
How could that happen? I can't think of any obvious scenarios.
> + kmem_start = res->start;
> + kmem_size = resource_size(res);
> + start_pfn = kmem_start >> PAGE_SHIFT;
> + end_pfn = start_pfn + (kmem_size >> PAGE_SHIFT) - 1;
> +
> + /*
> + * Walk and check that every singe memory_block of dax region is
> + * offline
> + */
> + lock_device_hotplug();
> + rc = walk_memory_range(start_pfn, end_pfn, dev,
> + check_memblock_offlined_cb);
Does lock_device_hotplug() also lock memory online/offline? Otherwise,
isn't this offline check racy? If not, can you please spell that out in
a comment?
Also, could you compare this a bit to the walk_memory_range() use in
__remove_memory()? Why do we need two walks looking for offline blocks?
next prev parent reply other threads:[~2019-04-25 19:01 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-25 17:54 [v3 0/2] "Hotremove" persistent memory Pavel Tatashin
2019-04-25 17:54 ` [v3 1/2] device-dax: fix memory and resource leak if hotplug fails Pavel Tatashin
2019-04-25 18:32 ` Dave Hansen
2019-04-25 18:51 ` Pavel Tatashin
2019-04-25 17:54 ` [v3 2/2] device-dax: "Hotremove" persistent memory that is used like normal RAM Pavel Tatashin
2019-04-25 19:01 ` Dave Hansen [this message]
2019-04-25 20:21 ` Pavel Tatashin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=77c286e3-8708-6e64-94a1-fb44b6bbff3f@intel.com \
--to=dave.hansen@intel.com \
--cc=akpm@linux-foundation.org \
--cc=baiyaowei@cmss.chinamobile.com \
--cc=bhelgaas@google.com \
--cc=bp@suse.de \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=dave.jiang@intel.com \
--cc=david@redhat.com \
--cc=fengguang.wu@intel.com \
--cc=jglisse@redhat.com \
--cc=jmorris@namei.org \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=mhocko@suse.com \
--cc=pasha.tatashin@soleen.com \
--cc=sashal@kernel.org \
--cc=thomas.lendacky@amd.com \
--cc=tiwai@suse.de \
--cc=vishal.l.verma@intel.com \
--cc=ying.huang@intel.com \
--cc=zwisler@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).