linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Sierra Guiza, Alejandro (Alex)" <alex.sierra@amd.com>
To: Ralph Campbell <rcampbell@nvidia.com>,
	Felix Kuehling <felix.kuehling@amd.com>,
	akpm@linux-foundation.org, linux-mm@kvack.org,
	linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org
Cc: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
	hch@lst.de, jgg@nvidia.com, jglisse@redhat.com
Subject: Re: [PATCH v6 02/13] mm: remove extra ZONE_DEVICE struct page refcount
Date: Thu, 19 Aug 2021 13:00:34 -0500	[thread overview]
Message-ID: <600a4c43-271d-df98-d3e0-301af0e8d0fe@amd.com> (raw)
In-Reply-To: <e155ed59-8c3c-4046-e731-f082ee4b10bb@nvidia.com>


On 8/18/2021 2:28 PM, Ralph Campbell wrote:
> On 8/17/21 5:35 PM, Felix Kuehling wrote:
>> Am 2021-08-17 um 8:01 p.m. schrieb Ralph Campbell:
>>> On 8/12/21 11:31 PM, Alex Sierra wrote:
>>>> From: Ralph Campbell <rcampbell@nvidia.com>
>>>>
>>>> ZONE_DEVICE struct pages have an extra reference count that
>>>> complicates the
>>>> code for put_page() and several places in the kernel that need to
>>>> check the
>>>> reference count to see that a page is not being used (gup, compaction,
>>>> migration, etc.). Clean up the code so the reference count doesn't
>>>> need to
>>>> be treated specially for ZONE_DEVICE.
>>>>
>>>> v2:
>>>> AS: merged this patch in linux 5.11 version
>>>>
>>>> v5:
>>>> AS: add condition at try_grab_page to check for the zone device type,
>>>> while
>>>> page ref counter is checked less/equal to zero. In case of device
>>>> zone, pages
>>>> ref counter are initialized to zero.
>>>>
>>>> Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
>>>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>>>> ---
>>>>    arch/powerpc/kvm/book3s_hv_uvmem.c     |  2 +-
>>>>    drivers/gpu/drm/nouveau/nouveau_dmem.c |  2 +-
>>>>    fs/dax.c                               |  4 +-
>>>>    include/linux/dax.h                    |  2 +-
>>>>    include/linux/memremap.h               |  7 +--
>>>>    include/linux/mm.h                     | 13 +----
>>>>    lib/test_hmm.c                         |  2 +-
>>>>    mm/internal.h                          |  8 +++
>>>>    mm/memremap.c                          | 68 
>>>> +++++++-------------------
>>>>    mm/migrate.c                           |  5 --
>>>>    mm/page_alloc.c                        |  3 ++
>>>>    mm/swap.c                              | 45 ++---------------
>>>>    12 files changed, 46 insertions(+), 115 deletions(-)
>>>>
>>> I haven't seen a response to the issues I raised back at v3 of this
>>> series.
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flinux-mm%2F4f6dd918-d79b-1aa7-3a4c-caa67ddc29bc%40nvidia.com%2F&amp;data=04%7C01%7Calex.sierra%40amd.com%7Cd2bd2d4fbf764528540908d9627e5dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637649117156919772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=P7FxYm%2BkJaCkMFa3OHtuKrPOn7SvytFRmYQdIzq7rN4%3D&amp;reserved=0 
>>>
>>>
>>>
>>> Did I miss something?
>> I think part of the response was that we did more testing. Alex added
>> support for DEVICE_GENERIC pages to test_hmm and he ran DAX tests
>> recommended by Theodore Tso. In that testing he ran into a WARN_ON_ONCE
>> about a zero page refcount in try_get_page. The fix is in the latest
>> version of patch 2. But it's already obsolete because John Hubbard is
>> about to remove that function altogether.
>>
>> I think the issues you raised were more uncertainty than known bugs. It
>> seems the fact that you can have DAX pages with 0 refcount is a feature
>> more than a bug.
>>
>> Regards,
>>    Felix
>
> Did you test on a system without CONFIG_ARCH_HAS_PTE_SPECIAL defined?
> In that case, mmap() of a DAX device will call insert_page() which calls
> get_page() which would trigger VM_BUG_ON_PAGE().
>
> I can believe it is OK for PTE_SPECIAL page table entries to have no
> struct page or that MEMORY_DEVICE_GENERIC struct pages be mapped with
> a zero reference count using insert_pfn().
Hi Ralph,
We have tried the DAX tests with and without CONFIG_ARCH_HAS_PTE_SPECIAL 
defined.
Apparently none of the tests touches that condition for a DAX device. Of 
course,
that doesn't mean it could happen.

Regards,
Alex S.

>
>
> I find it hard to believe that other MM developers don't see an issue
> with a struct page with refcount == 0 and mapcount == 1.
>
> I don't see where init_page_count() is being called for the
> MEMORY_DEVICE_GENERIC or MEMORY_DEVICE_PRIVATE struct pages the AMD
> driver allocates and passes to migrate_vma_setup().
> Looks like svm_migrate_get_vram_page() needs to call init_page_count()
> instead of get_page(). (I'm looking at branch 
> origin/alexsierrag/device_generic
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FRadeonOpenCompute%2FROCK-Kernel-Driver.git&amp;data=04%7C01%7Calex.sierra%40amd.com%7Cd2bd2d4fbf764528540908d9627e5dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637649117156919772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=IXe8HP2s8x5OdJdERBkGOYJCQk3iqCu5AYkwpDL8zec%3D&amp;reserved=0)
Yes, you're right. My bad. Thanks for catching this up. I didn't realize 
I was missing
to define CONFIG_DEBUG_VM on my build. Therefore this BUG was never caught.
It worked after I replaced get_pages by init_page_count at
svm_migrate_get_vram_page. However, I don't think this is the best way 
to fix it.
Ideally, get_pages call should work for device pages with ref count 
equal to 0
too. Otherwise, we could overwrite refcounter if someone else is 
grabbing the page
concurrently.
I was thinking to add a special condition in get_pages for dev pages. 
This could
also fix the insert_page -> get_page call from a DAX device.

Regards,
Alex S.
>
>
> Also, what about the other places where is_device_private_page() is 
> called?
> Don't they need to be updated to call is_device_page() instead?
> One of my goals for this patch was to remove special casing reference 
> counts
> for ZONE_DEVICE pages in rmap.c, etc.
Correct, is_device_private_page is still used in rmap, memcontrol and 
migrate.c files
Looks like rmap and memcontrol should be replaced by is_device_page 
function. However,
I still need test to validate this. For migrate.c is used in 
remove_migration_pte and
migrate_vma_insert_page, however these are specific conditions for 
private device type
Thanks for raise these questions, I think we're getting close.

Regards,
Alex S.
>
> I still think this patch needs an ACK from a FS/DAX maintainer.
>

  reply	other threads:[~2021-08-19 18:00 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-13  6:31 [PATCH v6 00/13] Support DEVICE_GENERIC memory in migrate_vma_* Alex Sierra
2021-08-13  6:31 ` [PATCH v6 01/13] ext4/xfs: add page refcount helper Alex Sierra
2021-08-15  9:01   ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 02/13] mm: remove extra ZONE_DEVICE struct page refcount Alex Sierra
2021-08-15 15:37   ` Christoph Hellwig
2021-08-15 20:40     ` John Hubbard
2021-08-16 18:56       ` Felix Kuehling
2021-08-20  6:33       ` Jerome Glisse
2021-08-18  0:01   ` Ralph Campbell
2021-08-18  0:35     ` Felix Kuehling
2021-08-18 19:28       ` Ralph Campbell
2021-08-19 18:00         ` Sierra Guiza, Alejandro (Alex) [this message]
2021-08-19 19:59           ` Felix Kuehling
2021-08-20  4:40             ` Christoph Hellwig
2021-08-20  7:17           ` Jerome Glisse
2021-08-20  4:56         ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 03/13] kernel: resource: lookup_resource as exported symbol Alex Sierra
2021-08-13  6:31 ` [PATCH v6 04/13] drm/amdkfd: add SPM support for SVM Alex Sierra
2021-08-15  9:10   ` Christoph Hellwig
2021-08-16 18:54     ` Felix Kuehling
2021-08-17  5:47       ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 05/13] drm/amdkfd: generic type as sys mem on migration to ram Alex Sierra
2021-08-15 15:38   ` Christoph Hellwig
2021-08-16 19:53     ` Sierra Guiza, Alejandro (Alex)
2021-08-16 22:06       ` Zeng, Oak
2021-08-17  0:42         ` Felix Kuehling
2021-08-17  5:49       ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 06/13] include/linux/mm.h: helpers to check zone device generic type Alex Sierra
2021-08-15  9:16   ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 07/13] mm: add generic type support to migrate_vma helpers Alex Sierra
2021-08-15  9:19   ` Christoph Hellwig
2021-08-13  6:31 ` [PATCH v6 08/13] mm: call pgmap->ops->page_free for DEVICE_GENERIC pages Alex Sierra
2021-08-15 15:40   ` Christoph Hellwig
2021-08-16 19:00     ` Felix Kuehling
2021-08-17  5:50       ` Christoph Hellwig
2021-08-17 15:44         ` Felix Kuehling
2021-08-20  5:05           ` Christoph Hellwig
2021-08-20  7:24             ` Jerome Glisse
2021-08-13  6:31 ` [PATCH v6 09/13] lib: test_hmm add ioctl to get zone device type Alex Sierra
2021-08-13  6:31 ` [PATCH v6 10/13] lib: test_hmm add module param for " Alex Sierra
2021-08-13  6:31 ` [PATCH v6 11/13] lib: add support for device generic type in test_hmm Alex Sierra
2021-08-13  6:31 ` [PATCH v6 12/13] tools: update hmm-test to support device generic type Alex Sierra
2021-08-13  6:31 ` [PATCH v6 13/13] tools: update test_hmm script to support SP config Alex Sierra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=600a4c43-271d-df98-d3e0-301af0e8d0fe@amd.com \
    --to=alex.sierra@amd.com \
    --cc=akpm@linux-foundation.org \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=felix.kuehling@amd.com \
    --cc=hch@lst.de \
    --cc=jgg@nvidia.com \
    --cc=jglisse@redhat.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=rcampbell@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).