From: "Zeng, Oak" <Oak.Zeng@amd.com>
To: "Sierra Guiza, Alejandro (Alex)" <Alex.Sierra@amd.com>,
Christoph Hellwig <hch@lst.de>
Cc: "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"Kuehling, Felix" <Felix.Kuehling@amd.com>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"rcampbell@nvidia.com" <rcampbell@nvidia.com>,
"linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>,
"linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>,
"jgg@nvidia.com" <jgg@nvidia.com>,
"jglisse@redhat.com" <jglisse@redhat.com>
Subject: Re: [PATCH v6 05/13] drm/amdkfd: generic type as sys mem on migration to ram
Date: Mon, 16 Aug 2021 22:06:38 +0000 [thread overview]
Message-ID: <771FC2D4-15CF-464B-96FF-837533A85886@amd.com> (raw)
In-Reply-To: <694ea624-9dc7-7a25-78a6-308ee0debaea@amd.com>
Regards,
Oak
On 2021-08-16, 3:53 PM, "amd-gfx on behalf of Sierra Guiza, Alejandro (Alex)" <amd-gfx-bounces@lists.freedesktop.org on behalf of alex.sierra@amd.com> wrote:
On 8/15/2021 10:38 AM, Christoph Hellwig wrote:
> On Fri, Aug 13, 2021 at 01:31:42AM -0500, Alex Sierra wrote:
>> migrate.vma = vma;
>> migrate.start = start;
>> migrate.end = end;
>> - migrate.flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE;
>> migrate.pgmap_owner = SVM_ADEV_PGMAP_OWNER(adev);
>>
>> + if (adev->gmc.xgmi.connected_to_cpu)
>> + migrate.flags = MIGRATE_VMA_SELECT_SYSTEM;
>> + else
>> + migrate.flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE;
> It's been a while since I touched this migrate code, but doesn't this
> mean that if the range already contains system memory the migration
> now won't do anything? for the connected_to_cpu case?
For above’s condition equal to connected_to_cpu , we’re explicitly
migrating from
device memory to system memory with device generic type.
For MEMORY_DEVICE_GENERIC memory type, why do we need to explicitly migrate it from device memory to normal system memory? I thought the design was, for this type of memory, CPU can access it in place without migration(just like CPU access normal system memory), so there is no need to migrate such type of memory to normal system memory...
With this patch, the migration behavior will be: when memory is accessed by CPU, it will be migrated to normal system memory; when memory is accessed by GPU, it will be migrated to device vram. This is basically the same behavior as when vram is treated as DEVICE_PRIVATE.
I thought the whole goal of introducing DEVICE_GENERIC is to avoid such back and forth migration b/t device memory and normal system memory. But maybe I am missing something here....
Regards,
Oak
In this type,
device PTEs are
present in CPU page table.
During migrate_vma_collect_pmd walk op at migrate_vma_setup call,
there’s a condition
for present pte that require migrate->flags be set for
MIGRATE_VMA_SELECT_SYSTEM.
Otherwise, the migration for this entry will be ignored.
Regards,
Alex S.
next prev parent reply other threads:[~2021-08-16 22:06 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-13 6:31 [PATCH v6 00/13] Support DEVICE_GENERIC memory in migrate_vma_* Alex Sierra
2021-08-13 6:31 ` [PATCH v6 01/13] ext4/xfs: add page refcount helper Alex Sierra
2021-08-15 9:01 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 02/13] mm: remove extra ZONE_DEVICE struct page refcount Alex Sierra
2021-08-15 15:37 ` Christoph Hellwig
2021-08-15 20:40 ` John Hubbard
2021-08-16 18:56 ` Felix Kuehling
2021-08-20 6:33 ` Jerome Glisse
2021-08-18 0:01 ` Ralph Campbell
2021-08-18 0:35 ` Felix Kuehling
2021-08-18 19:28 ` Ralph Campbell
2021-08-19 18:00 ` Sierra Guiza, Alejandro (Alex)
2021-08-19 19:59 ` Felix Kuehling
2021-08-20 4:40 ` Christoph Hellwig
2021-08-20 7:17 ` Jerome Glisse
2021-08-20 4:56 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 03/13] kernel: resource: lookup_resource as exported symbol Alex Sierra
2021-08-13 6:31 ` [PATCH v6 04/13] drm/amdkfd: add SPM support for SVM Alex Sierra
2021-08-15 9:10 ` Christoph Hellwig
2021-08-16 18:54 ` Felix Kuehling
2021-08-17 5:47 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 05/13] drm/amdkfd: generic type as sys mem on migration to ram Alex Sierra
2021-08-15 15:38 ` Christoph Hellwig
2021-08-16 19:53 ` Sierra Guiza, Alejandro (Alex)
2021-08-16 22:06 ` Zeng, Oak [this message]
2021-08-17 0:42 ` Felix Kuehling
2021-08-17 5:49 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 06/13] include/linux/mm.h: helpers to check zone device generic type Alex Sierra
2021-08-15 9:16 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 07/13] mm: add generic type support to migrate_vma helpers Alex Sierra
2021-08-15 9:19 ` Christoph Hellwig
2021-08-13 6:31 ` [PATCH v6 08/13] mm: call pgmap->ops->page_free for DEVICE_GENERIC pages Alex Sierra
2021-08-15 15:40 ` Christoph Hellwig
2021-08-16 19:00 ` Felix Kuehling
2021-08-17 5:50 ` Christoph Hellwig
2021-08-17 15:44 ` Felix Kuehling
2021-08-20 5:05 ` Christoph Hellwig
2021-08-20 7:24 ` Jerome Glisse
2021-08-13 6:31 ` [PATCH v6 09/13] lib: test_hmm add ioctl to get zone device type Alex Sierra
2021-08-13 6:31 ` [PATCH v6 10/13] lib: test_hmm add module param for " Alex Sierra
2021-08-13 6:31 ` [PATCH v6 11/13] lib: add support for device generic type in test_hmm Alex Sierra
2021-08-13 6:31 ` [PATCH v6 12/13] tools: update hmm-test to support device generic type Alex Sierra
2021-08-13 6:31 ` [PATCH v6 13/13] tools: update test_hmm script to support SP config Alex Sierra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=771FC2D4-15CF-464B-96FF-837533A85886@amd.com \
--to=oak.zeng@amd.com \
--cc=Alex.Sierra@amd.com \
--cc=Felix.Kuehling@amd.com \
--cc=akpm@linux-foundation.org \
--cc=amd-gfx@lists.freedesktop.org \
--cc=dri-devel@lists.freedesktop.org \
--cc=hch@lst.de \
--cc=jgg@nvidia.com \
--cc=jglisse@redhat.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
--cc=rcampbell@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).