From: Ralph Campbell <rcampbell@nvidia.com>
To: Felix Kuehling <felix.kuehling@amd.com>,
Alex Sierra <alex.sierra@amd.com>, <akpm@linux-foundation.org>,
<linux-mm@kvack.org>, <linux-ext4@vger.kernel.org>,
<linux-xfs@vger.kernel.org>
Cc: <amd-gfx@lists.freedesktop.org>,
<dri-devel@lists.freedesktop.org>, <hch@lst.de>, <jgg@nvidia.com>,
<jglisse@redhat.com>
Subject: Re: [PATCH v6 02/13] mm: remove extra ZONE_DEVICE struct page refcount
Date: Wed, 18 Aug 2021 12:28:30 -0700 [thread overview]
Message-ID: <e155ed59-8c3c-4046-e731-f082ee4b10bb@nvidia.com> (raw)
In-Reply-To: <393e9815-838d-5fe6-d6ab-bfe7b543fef6@amd.com>
On 8/17/21 5:35 PM, Felix Kuehling wrote:
> Am 2021-08-17 um 8:01 p.m. schrieb Ralph Campbell:
>> On 8/12/21 11:31 PM, Alex Sierra wrote:
>>> From: Ralph Campbell <rcampbell@nvidia.com>
>>>
>>> ZONE_DEVICE struct pages have an extra reference count that
>>> complicates the
>>> code for put_page() and several places in the kernel that need to
>>> check the
>>> reference count to see that a page is not being used (gup, compaction,
>>> migration, etc.). Clean up the code so the reference count doesn't
>>> need to
>>> be treated specially for ZONE_DEVICE.
>>>
>>> v2:
>>> AS: merged this patch in linux 5.11 version
>>>
>>> v5:
>>> AS: add condition at try_grab_page to check for the zone device type,
>>> while
>>> page ref counter is checked less/equal to zero. In case of device
>>> zone, pages
>>> ref counter are initialized to zero.
>>>
>>> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
>>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>>> ---
>>> arch/powerpc/kvm/book3s_hv_uvmem.c | 2 +-
>>> drivers/gpu/drm/nouveau/nouveau_dmem.c | 2 +-
>>> fs/dax.c | 4 +-
>>> include/linux/dax.h | 2 +-
>>> include/linux/memremap.h | 7 +--
>>> include/linux/mm.h | 13 +----
>>> lib/test_hmm.c | 2 +-
>>> mm/internal.h | 8 +++
>>> mm/memremap.c | 68 +++++++-------------------
>>> mm/migrate.c | 5 --
>>> mm/page_alloc.c | 3 ++
>>> mm/swap.c | 45 ++---------------
>>> 12 files changed, 46 insertions(+), 115 deletions(-)
>>>
>> I haven't seen a response to the issues I raised back at v3 of this
>> series.
>> https://lore.kernel.org/linux-mm/4f6dd918-d79b-1aa7-3a4c-caa67ddc29bc@nvidia.com/
>>
>>
>> Did I miss something?
> I think part of the response was that we did more testing. Alex added
> support for DEVICE_GENERIC pages to test_hmm and he ran DAX tests
> recommended by Theodore Tso. In that testing he ran into a WARN_ON_ONCE
> about a zero page refcount in try_get_page. The fix is in the latest
> version of patch 2. But it's already obsolete because John Hubbard is
> about to remove that function altogether.
>
> I think the issues you raised were more uncertainty than known bugs. It
> seems the fact that you can have DAX pages with 0 refcount is a feature
> more than a bug.
>
> Regards,
> Felix
Did you test on a system without CONFIG_ARCH_HAS_PTE_SPECIAL defined?
In that case, mmap() of a DAX device will call insert_page() which calls
get_page() which would trigger VM_BUG_ON_PAGE().
I can believe it is OK for PTE_SPECIAL page table entries to have no
struct page or that MEMORY_DEVICE_GENERIC struct pages be mapped with
a zero reference count using insert_pfn().
I find it hard to believe that other MM developers don't see an issue
with a struct page with refcount == 0 and mapcount == 1.
I don't see where init_page_count() is being called for the
MEMORY_DEVICE_GENERIC or MEMORY_DEVICE_PRIVATE struct pages the AMD
driver allocates and passes to migrate_vma_setup().
Looks like svm_migrate_get_vram_page() needs to call init_page_count()
instead of get_page(). (I'm looking at branch origin/alexsierrag/device_generic
https://github.com/RadeonOpenCompute/ROCK-Kernel-Driver.git)
Also, what about the other places where is_device_private_page() is called?
Don't they need to be updated to call is_device_page() instead?
One of my goals for this patch was to remove special casing reference counts
for ZONE_DEVICE pages in rmap.c, etc.
I still think this patch needs an ACK from a FS/DAX maintainer.
next prev parent reply other threads:[~2021-08-18 19:28 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-13 6:31 [PATCH v6 00/13] Support DEVICE_GENERIC memory in migrate_vma_* Alex Sierra
2021-08-13 6:31 ` [PATCH v6 01/13] ext4/xfs: add page refcount helper Alex Sierra
2021-08-15 9:01 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 02/13] mm: remove extra ZONE_DEVICE struct page refcount Alex Sierra
2021-08-15 15:37 ` Christoph Hellwig
2021-08-15 20:40 ` John Hubbard
2021-08-16 18:56 ` Felix Kuehling
2021-08-20 6:33 ` Jerome Glisse
2021-08-18 0:01 ` Ralph Campbell
2021-08-18 0:35 ` Felix Kuehling
2021-08-18 19:28 ` Ralph Campbell [this message]
2021-08-19 18:00 ` Sierra Guiza, Alejandro (Alex)
2021-08-19 19:59 ` Felix Kuehling
2021-08-20 4:40 ` Christoph Hellwig
2021-08-20 7:17 ` Jerome Glisse
2021-08-20 4:56 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 03/13] kernel: resource: lookup_resource as exported symbol Alex Sierra
2021-08-13 6:31 ` [PATCH v6 04/13] drm/amdkfd: add SPM support for SVM Alex Sierra
2021-08-15 9:10 ` Christoph Hellwig
2021-08-16 18:54 ` Felix Kuehling
2021-08-17 5:47 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 05/13] drm/amdkfd: generic type as sys mem on migration to ram Alex Sierra
2021-08-15 15:38 ` Christoph Hellwig
2021-08-16 19:53 ` Sierra Guiza, Alejandro (Alex)
2021-08-16 22:06 ` Zeng, Oak
2021-08-17 0:42 ` Felix Kuehling
2021-08-17 5:49 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 06/13] include/linux/mm.h: helpers to check zone device generic type Alex Sierra
2021-08-15 9:16 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 07/13] mm: add generic type support to migrate_vma helpers Alex Sierra
2021-08-15 9:19 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 08/13] mm: call pgmap->ops->page_free for DEVICE_GENERIC pages Alex Sierra
2021-08-15 15:40 ` Christoph Hellwig
2021-08-16 19:00 ` Felix Kuehling
2021-08-17 5:50 ` Christoph Hellwig
2021-08-17 15:44 ` Felix Kuehling
2021-08-20 5:05 ` Christoph Hellwig
2021-08-20 7:24 ` Jerome Glisse
2021-08-13 6:31 ` [PATCH v6 09/13] lib: test_hmm add ioctl to get zone device type Alex Sierra
2021-08-13 6:31 ` [PATCH v6 10/13] lib: test_hmm add module param for " Alex Sierra
2021-08-13 6:31 ` [PATCH v6 11/13] lib: add support for device generic type in test_hmm Alex Sierra
2021-08-13 6:31 ` [PATCH v6 12/13] tools: update hmm-test to support device generic type Alex Sierra
2021-08-13 6:31 ` [PATCH v6 13/13] tools: update test_hmm script to support SP config Alex Sierra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e155ed59-8c3c-4046-e731-f082ee4b10bb@nvidia.com \
--to=rcampbell@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=alex.sierra@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=felix.kuehling@amd.com \
--cc=hch@lst.de \
--cc=jgg@nvidia.com \
--cc=jglisse@redhat.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).