nvdimm.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: Joao Martins <joao.m.martins@oracle.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: Linux MM <linux-mm@kvack.org>,
	Vishal Verma <vishal.l.verma@intel.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Naoya Horiguchi <naoya.horiguchi@nec.com>,
	Matthew Wilcox <willy@infradead.org>,
	Jason Gunthorpe <jgg@ziepe.ca>,
	John Hubbard <jhubbard@nvidia.com>,
	Jane Chu <jane.chu@oracle.com>,
	Muchun Song <songmuchun@bytedance.com>,
	Mike Kravetz <mike.kravetz@oracle.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jonathan Corbet <corbet@lwn.net>, Christoph Hellwig <hch@lst.de>,
	Linux NVDIMM <nvdimm@lists.linux.dev>,
	Linux Doc Mailing List <linux-doc@vger.kernel.org>
Subject: Re: [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices
Date: Fri, 5 Nov 2021 12:09:19 +0000	[thread overview]
Message-ID: <f33c2037-4bee-3564-75c0-c87f99325c02@oracle.com> (raw)
In-Reply-To: <CAPcyv4hPV9Vur1uvga7S4krQAmKZK5jrBrdOuK1AFHVE8Zk1DA@mail.gmail.com>

On 11/5/21 00:31, Dan Williams wrote:
> On Fri, Aug 27, 2021 at 7:59 AM Joao Martins <joao.m.martins@oracle.com> wrote:
>>
>> Right now, only static dax regions have a valid @pgmap pointer in its
>> struct dev_dax. Dynamic dax case however, do not.
>>
>> In preparation for device-dax compound devmap support, make sure that
>> dev_dax pgmap field is set after it has been allocated and initialized.
>>
>> dynamic dax device have the @pgmap is allocated at probe() and it's
>> managed by devm (contrast to static dax region which a pgmap is provided
>> and dax core kfrees it). So in addition to ensure a valid @pgmap, clear
>> the pgmap when the dynamic dax device is released to avoid the same
>> pgmap ranges to be re-requested across multiple region device reconfigs.
>>
>> Suggested-by: Dan Williams <dan.j.williams@intel.com>
>> Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
>> ---
>>  drivers/dax/bus.c    | 8 ++++++++
>>  drivers/dax/device.c | 2 ++
>>  2 files changed, 10 insertions(+)
>>
>> diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
>> index 6cc4da4c713d..49dbff9ba609 100644
>> --- a/drivers/dax/bus.c
>> +++ b/drivers/dax/bus.c
>> @@ -363,6 +363,14 @@ void kill_dev_dax(struct dev_dax *dev_dax)
>>
>>         kill_dax(dax_dev);
>>         unmap_mapping_range(inode->i_mapping, 0, 0, 1);
>> +
>> +       /*
>> +        * Dynamic dax region have the pgmap allocated via dev_kzalloc()
>> +        * and thus freed by devm. Clear the pgmap to not have stale pgmap
>> +        * ranges on probe() from previous reconfigurations of region devices.
>> +        */
>> +       if (!is_static(dev_dax->region))
>> +               dev_dax->pgmap = NULL;
>>  }
>>  EXPORT_SYMBOL_GPL(kill_dev_dax);
>>
>> diff --git a/drivers/dax/device.c b/drivers/dax/device.c
>> index 0b82159b3564..6e348b5f9d45 100644
>> --- a/drivers/dax/device.c
>> +++ b/drivers/dax/device.c
>> @@ -426,6 +426,8 @@ int dev_dax_probe(struct dev_dax *dev_dax)
>>         }
>>
>>         pgmap->type = MEMORY_DEVICE_GENERIC;
>> +       dev_dax->pgmap = pgmap;
> 
> So I think I'd rather see a bigger patch that replaces some of the
> implicit dev_dax->pgmap == NULL checks with explicit is_static()
> checks. Something like the following only compile and boot tested...
> Note the struct_size() change probably wants to be its own cleanup,
> and the EXPORT_SYMBOL_NS_GPL(..., DAX) probably wants to be its own
> patch converting over the entirety of drivers/dax/. Thoughts?
> 
It's a good idea. Certainly the implicit pgmap == NULL made it harder
than the necessary to find where the problem was. So turning those checks
into explicit checks that differentiate static vs dynamic dax will help

With respect to this series converting those pgmap == NULL is going to need
to made me export the symbol (provided dax core and dax device can be built
as modules). So I don't know how this can be a patch converting entirety of
dax. Perhaps you mean that I would just EXPORT_SYMBOL() and then a bigger
patch introduces the MODULE_NS_IMPORT() And EXPORT_SYMBOL_NS*() separately.

The struct_size, yeah, should be a separate patch much like commit 7d18dd75a8af
("device-dax/kmem: use struct_size()").

minor comment below on your snippet.

> 
> diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
> index 6cc4da4c713d..67ab7e05b340 100644
> --- a/drivers/dax/bus.c
> +++ b/drivers/dax/bus.c
> @@ -134,6 +134,12 @@ static bool is_static(struct dax_region *dax_region)
>         return (dax_region->res.flags & IORESOURCE_DAX_STATIC) != 0;
>  }
> 
> +bool static_dev_dax(struct dev_dax *dev_dax)
> +{
> +       return is_static(dev_dax->region);
> +}
> +EXPORT_SYMBOL_NS_GPL(static_dev_dax, DAX);
> +
>  static u64 dev_dax_size(struct dev_dax *dev_dax)
>  {
>         u64 size = 0;
> @@ -363,6 +369,8 @@ void kill_dev_dax(struct dev_dax *dev_dax)
> 
>         kill_dax(dax_dev);
>         unmap_mapping_range(inode->i_mapping, 0, 0, 1);
> +       if (static_dev_dax(dev_dax))
> +               dev_dax->pgmap = NULL;
>  }

Here you probably meant !static_dev_dax() per my patch.

>  EXPORT_SYMBOL_GPL(kill_dev_dax);
> 
> diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h
> index 1e946ad7780a..4acdfee7dd59 100644
> --- a/drivers/dax/bus.h
> +++ b/drivers/dax/bus.h
> @@ -48,6 +48,7 @@ int __dax_driver_register(struct dax_device_driver *dax_drv,
>         __dax_driver_register(driver, THIS_MODULE, KBUILD_MODNAME)
>  void dax_driver_unregister(struct dax_device_driver *dax_drv);
>  void kill_dev_dax(struct dev_dax *dev_dax);
> +bool static_dev_dax(struct dev_dax *dev_dax);
> 
>  #if IS_ENABLED(CONFIG_DEV_DAX_PMEM_COMPAT)
>  int dev_dax_probe(struct dev_dax *dev_dax);
> diff --git a/drivers/dax/device.c b/drivers/dax/device.c
> index dd8222a42808..87507aff2b10 100644
> --- a/drivers/dax/device.c
> +++ b/drivers/dax/device.c
> @@ -398,31 +398,43 @@ int dev_dax_probe(struct dev_dax *dev_dax)
>         void *addr;
>         int rc, i;
> 
> -       pgmap = dev_dax->pgmap;
> -       if (dev_WARN_ONCE(dev, pgmap && dev_dax->nr_range > 1,
> -                       "static pgmap / multi-range device conflict\n"))
> +       if (static_dev_dax(dev_dax) && dev_dax->nr_range > 1) {
> +               dev_warn(dev, "static pgmap / multi-range device conflict\n");
>                 return -EINVAL;
> +       }
> 
> -       if (!pgmap) {
> -               pgmap = devm_kzalloc(dev, sizeof(*pgmap) + sizeof(struct range)
> -                               * (dev_dax->nr_range - 1), GFP_KERNEL);
> +       if (static_dev_dax(dev_dax)) {
> +               pgmap = dev_dax->pgmap;
> +       } else {
> +               if (dev_dax->pgmap) {
> +                       dev_warn(dev,
> +                                "dynamic-dax with pre-populated page map!?\n");
> +                       return -EINVAL;
> +               }
> +               pgmap = devm_kzalloc(
> +                       dev, struct_size(pgmap, ranges, dev_dax->nr_range - 1),
> +                       GFP_KERNEL);
>                 if (!pgmap)
>                         return -ENOMEM;
>                 pgmap->nr_range = dev_dax->nr_range;
> +               dev_dax->pgmap = pgmap;
> +               for (i = 0; i < dev_dax->nr_range; i++) {
> +                       struct range *range = &dev_dax->ranges[i].range;
> +
> +                       pgmap->ranges[i] = *range;
> +               }
>         }
> 
This code move is probably not needed unless your point is to have a more clear
separation on what's initialization versus the mem region request (that's
applicable to both dynamic and static).

>         for (i = 0; i < dev_dax->nr_range; i++) {
>                 struct range *range = &dev_dax->ranges[i].range;
> 
> -               if (!devm_request_mem_region(dev, range->start,
> -                                       range_len(range), dev_name(dev))) {
> -                       dev_warn(dev, "mapping%d: %#llx-%#llx could
> not reserve range\n",
> -                                       i, range->start, range->end);
> -                       return -EBUSY;
> -               }
> -               /* don't update the range for static pgmap */
> -               if (!dev_dax->pgmap)
> -                       pgmap->ranges[i] = *range;
> +               if (devm_request_mem_region(dev, range->start, range_len(range),
> +                                           dev_name(dev)))
> +                       continue;
> +               dev_warn(dev,
> +                        "mapping%d: %#llx-%#llx could not reserve range\n", i,
> +                        range->start, range->end);
> +               return -EBUSY;
>         }
> 
>         pgmap->type = MEMORY_DEVICE_GENERIC;
> @@ -473,3 +485,4 @@ MODULE_LICENSE("GPL v2");
>  module_init(dax_init);
>  module_exit(dax_exit);
>  MODULE_ALIAS_DAX_DEVICE(0);
> +MODULE_IMPORT_NS(DAX);


  reply	other threads:[~2021-11-05 12:10 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-27 14:58 [PATCH v4 00/14] mm, sparse-vmemmap: Introduce compound devmaps for device-dax Joao Martins
2021-08-27 14:58 ` [PATCH v4 01/14] memory-failure: fetch compound_head after pgmap_pfn_valid() Joao Martins
2021-08-27 14:58 ` [PATCH v4 02/14] mm/page_alloc: split prep_compound_page into head and tail subparts Joao Martins
2021-08-27 14:58 ` [PATCH v4 03/14] mm/page_alloc: refactor memmap_init_zone_device() page init Joao Martins
2021-08-27 14:58 ` [PATCH v4 04/14] mm/memremap: add ZONE_DEVICE support for compound pages Joao Martins
2021-08-27 15:33   ` Christoph Hellwig
2021-08-27 16:00     ` Joao Martins
2021-09-01  9:44       ` Christoph Hellwig
2021-09-09  9:38         ` Joao Martins
2021-08-27 14:58 ` [PATCH v4 05/14] device-dax: use ALIGN() for determining pgoff Joao Martins
2021-08-27 14:58 ` [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices Joao Martins
2021-11-05  0:31   ` Dan Williams
2021-11-05 12:09     ` Joao Martins [this message]
2021-11-05 16:14       ` Joao Martins
2021-11-05 16:46       ` Dan Williams
2021-11-05 18:11         ` Joao Martins
2021-08-27 14:58 ` [PATCH v4 07/14] device-dax: compound devmap support Joao Martins
2021-11-05  0:38   ` Dan Williams
2021-11-05 14:10     ` Joao Martins
2021-11-05 16:41       ` Dan Williams
2021-08-27 14:58 ` [PATCH v4 08/14] mm/gup: grab head page refcount once for group of subpages Joao Martins
2021-08-27 16:25   ` Jason Gunthorpe
2021-08-27 18:34     ` Joao Martins
2021-08-30 13:07       ` Jason Gunthorpe
2021-08-31 12:34         ` Joao Martins
2021-08-31 17:05           ` Jason Gunthorpe
2021-09-23 16:51             ` Joao Martins
2021-09-28 18:01               ` Jason Gunthorpe
2021-09-29 11:50                 ` Joao Martins
2021-09-29 19:34                   ` Jason Gunthorpe
2021-09-30  3:01                     ` Alistair Popple
2021-09-30 17:54                       ` Joao Martins
2021-09-30 21:55                         ` Jason Gunthorpe
2021-10-18 18:36                       ` Jason Gunthorpe
2021-10-18 18:37                   ` Jason Gunthorpe
2021-10-08 11:54   ` Jason Gunthorpe
2021-10-11 15:53     ` Joao Martins
2021-10-13 17:41       ` Jason Gunthorpe
2021-10-13 19:18         ` Joao Martins
2021-10-13 19:43           ` Jason Gunthorpe
2021-10-14 17:56             ` Joao Martins
2021-10-14 18:06               ` Jason Gunthorpe
2021-08-27 14:58 ` [PATCH v4 09/14] mm/sparse-vmemmap: add a pgmap argument to section activation Joao Martins
2021-08-27 14:58 ` [PATCH v4 10/14] mm/sparse-vmemmap: refactor core of vmemmap_populate_basepages() to helper Joao Martins
2021-08-27 14:58 ` [PATCH v4 11/14] mm/hugetlb_vmemmap: move comment block to Documentation/vm Joao Martins
2021-08-27 14:58 ` [PATCH v4 12/14] mm/sparse-vmemmap: populate compound devmaps Joao Martins
2021-08-27 14:58 ` [PATCH v4 13/14] mm/page_alloc: reuse tail struct pages for " Joao Martins
2021-08-27 14:58 ` [PATCH v4 14/14] mm/sparse-vmemmap: improve memory savings for compound pud geometry Joao Martins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f33c2037-4bee-3564-75c0-c87f99325c02@oracle.com \
    --to=joao.m.martins@oracle.com \
    --cc=akpm@linux-foundation.org \
    --cc=corbet@lwn.net \
    --cc=dan.j.williams@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=hch@lst.de \
    --cc=jane.chu@oracle.com \
    --cc=jgg@ziepe.ca \
    --cc=jhubbard@nvidia.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mike.kravetz@oracle.com \
    --cc=naoya.horiguchi@nec.com \
    --cc=nvdimm@lists.linux.dev \
    --cc=songmuchun@bytedance.com \
    --cc=vishal.l.verma@intel.com \
    --cc=willy@infradead.org \
    --subject='Re: [PATCH v4 06/14] device-dax: ensure dev_dax->pgmap is valid for dynamic devices' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).