linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Logan Gunthorpe <logang@deltatee.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: "Andrew Morton" <akpm@linux-foundation.org>,
	stable <stable@vger.kernel.org>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Christoph Hellwig" <hch@lst.de>,
	"Linus Torvalds" <torvalds@linux-foundation.org>,
	"Linux MM" <linux-mm@kvack.org>,
	"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
	"Maling list - DRI developers" <dri-devel@lists.freedesktop.org>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Stephen Bates" <sbates@raithlin.com>
Subject: Re: [PATCH v8 3/7] mm, devm_memremap_pages: Fix shutdown handling
Date: Thu, 29 Nov 2018 10:50:46 -0700	[thread overview]
Message-ID: <43778343-6d43-eb43-0de0-3db6828902d0@deltatee.com> (raw)
In-Reply-To: <CAPcyv4gZisOAE8VJPJChNXrWv0NhUevWuutsPdvNORBTOBXJfA@mail.gmail.com>



On 2018-11-29 10:30 a.m., Dan Williams wrote:
> Oh! Yes, nice find. We need to wait for the percpu-ref to be dead and
> all outstanding references dropped before we can proceed to
> arch_remove_memory(), and I think this problem has been there since
> day one because the final exit was always after devm_memremap_pages()
> release which means arch_remove_memory() was always racing any final
> put_page(). I'll take a look, it seems the arch_remove_pages() call
> needs to be moved out-of-line to its own context and wait for the
> final exit of the percpu-ref.

Ok, well I thought moving the wait_for_completion() into the kill() call
was a pretty good solution to this. Though, if we move the
arch_remove_pages() into a different context, it *may* help with the
problem below...

>> Though, now that I look at it, the current change in question will be
>> wrong if there are two devm_memremap_pages_release()s to call. Both need
>> to drop their references before we can wait_for_completion() ;(. I guess
>> I need multiple percpu_refs or more complex changes to
>> devm_memremap_pages_release().
> 
> Can you just have a normal device-level kref for this case? On final
> device-level kref_put then kill the percpu_ref? I guess the problem is
> devm semantics where p2pdma only gets one callback on a driver
> ->remove() event. I'm not sure how to support multiple references of
> the same pages without creating a non-devm version of
> devm_memremap_pages(). I'm not opposed to that, but afaiu I don't
> think p2pdma is compatible with devm as long as it supports N>1:1
> mappings of the same range.

Hmm, no I think you misunderstood what I said. I'm saying I need to have
exactly one percpu_ref per call to devm_memremap_pages() and this is
doable, just slightly annoying. Right now I have one percpu_ref for
multiple calls to devm_memremap_pages() which doesn't work with the
above fix because there will always be a wait_for_completion() before
the last references are dropped in this way:

1) First devm_memremap_pages_release() is called which drops it's
reference and waits_for_completion().

2) The second devm_memremap_pages_release() needs to be called to drop
it's reference, but can't seeing the first is waiting, and therefore the
percpu_ref never goes to zero and the wait_for_completion() never returns.

Logan

  reply	other threads:[~2018-11-29 17:51 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-20 23:12 [PATCH v8 0/7] mm: Merge hmm into devm_memremap_pages, mark GPL-only Dan Williams
2018-11-20 23:12 ` [PATCH v8 1/7] mm, devm_memremap_pages: Mark devm_memremap_pages() EXPORT_SYMBOL_GPL Dan Williams
2018-11-22 13:30   ` Michal Hocko
2018-11-22 16:38     ` Christoph Hellwig
2018-11-22 16:40       ` Christoph Hellwig
2018-11-23  8:47       ` Michal Hocko
2018-11-20 23:13 ` [PATCH v8 2/7] mm, devm_memremap_pages: Kill mapping "System RAM" support Dan Williams
2018-11-20 23:13 ` [PATCH v8 3/7] mm, devm_memremap_pages: Fix shutdown handling Dan Williams
2018-11-27 21:43   ` Logan Gunthorpe
2018-11-29  3:10     ` Dan Williams
2018-11-29 17:06       ` Logan Gunthorpe
2018-11-29 17:30         ` Dan Williams
2018-11-29 17:50           ` Logan Gunthorpe [this message]
2018-11-29 18:51             ` Dan Williams
2018-11-30 22:19               ` Logan Gunthorpe
2018-11-30 22:28                 ` Dan Williams
2018-11-30 22:34                   ` Logan Gunthorpe
2018-11-30 22:47                     ` Dan Williams
2018-11-20 23:13 ` [PATCH v8 4/7] mm, devm_memremap_pages: Add MEMORY_DEVICE_PRIVATE support Dan Williams
2018-11-23 10:48   ` David Hildenbrand
2018-11-20 23:13 ` [PATCH v8 5/7] mm, hmm: Use devm semantics for hmm_devmem_{add, remove} Dan Williams
2018-11-20 23:13 ` [PATCH v8 6/7] mm, hmm: Replace hmm_devmem_pages_create() with devm_memremap_pages() Dan Williams
2018-11-20 23:13 ` [PATCH v8 7/7] mm, hmm: Mark hmm_devmem_{add, add_resource} EXPORT_SYMBOL_GPL Dan Williams
2018-11-22  1:20 ` [PATCH v8 0/7] mm: Merge hmm into devm_memremap_pages, mark GPL-only Andrew Morton
2018-11-25 22:04   ` Pavel Machek
2018-12-03 23:37 ` Jerome Glisse
2019-02-10 11:09 [PATCH v8 3/7] mm, devm_memremap_pages: Fix shutdown handling Krzysztof Grygiencz
2019-02-11 19:57 ` Jerome Glisse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=43778343-6d43-eb43-0de0-3db6828902d0@deltatee.com \
    --to=logang@deltatee.com \
    --cc=akpm@linux-foundation.org \
    --cc=bhelgaas@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=hch@lst.de \
    --cc=jglisse@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=sbates@raithlin.com \
    --cc=stable@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: [PATCH v8 3/7] mm, devm_memremap_pages: Fix shutdown handling' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).