Nouveau Archive on lore.kernel.org
 help / color / Atom feed
From: Alistair Popple <apopple@nvidia.com>
To: <linux-mm@kvack.org>, <nouveau@lists.freedesktop.org>,
	<bskeggs@redhat.com>, <akpm@linux-foundation.org>
Cc: rcampbell@nvidia.com, linux-doc@vger.kernel.org,
	Alistair Popple <apopple@nvidia.com>,
	linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org,
	kvm-ppc@vger.kernel.org
Subject: [Nouveau] [PATCH 0/9] Add support for SVM atomics in Nouveau
Date: Tue, 9 Feb 2021 12:07:13 +1100
Message-ID: <20210209010722.13839-1-apopple@nvidia.com> (raw)

This series adds support to Nouveau for atomic memory operations on OpenCL
shared virtual memory (SVM). This is achieved using the atomic PTE bits on
the GPU to only permit atomic operations to system memory when a page is
not mapped in userspace on the CPU.

This is implemented by adding a mode to migrate_vma_pages() which unmaps
and isolates existing pages from the CPU and pins them. The original
userspace page table entries are migrated to point to device private pages
allocated by the driver. This allows the driver to enable GPU atomic access
to the page as it will receive a callback when CPU userspace needs to
access it.

In response to this callback the driver revokes the atomic access
permission from the GPU and migrates entries to point back to the original
page. The original page is unpinned as part of the migration operation
which also returns it to the LRU.

Patch 3 contains the bulk of the memory management changes to implement
unmap and pin.

Patches 6-9 extend Nouveau to use the new mode to allow system wide atomics
for OpenCL SVM to be implemented on Nouveau.

This has been tested using the latest upstream Mesa userspace with a simple
OpenCL test program which checks the results of atomic GPU operations on a
buffer whilst also writing to the same buffer from the CPU.

Problems yet to be addressed:

Recent changes to pin_user_pages() prevent the creation of pinned pages in
ZONE_MOVABLE. This series allows pinned pages to be created in ZONE_MOVABLE
as attempts to migrate may fail which would be fatal to userspace.

In this case migration of the pinned page is unnecessary as the page can be
unpinned at anytime by having the driver revoke atomic permission as it
does for the migrate_to_ram() callback. However a method of calling this
when memory needs to be moved has yet to be resolved so any discussion is
welcome.

Alistair Popple (9):
  mm/migrate.c: Always allow device private pages to migrate
  mm/migrate.c: Allow pfn flags to be passed to migrate_vma_setup()
  mm/migrate: Add a unmap and pin migration mode
  Documentation: Add unmap and pin to HMM
  hmm-tests: Add test for unmap and pin
  nouveau/dmem: Only map migrating pages
  nouveau/svm: Refactor nouveau_range_fault
  nouveau/dmem: Add support for multiple page types
  nouveau/svm: Implement atomic SVM access

 Documentation/vm/hmm.rst                      |  22 +-
 arch/powerpc/kvm/book3s_hv_uvmem.c            |   4 +-
 drivers/gpu/drm/nouveau/include/nvif/if000c.h |   1 +
 drivers/gpu/drm/nouveau/nouveau_dmem.c        | 190 +++++++++++++++---
 drivers/gpu/drm/nouveau/nouveau_dmem.h        |   9 +
 drivers/gpu/drm/nouveau/nouveau_svm.c         | 148 +++++++++++---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h |   1 +
 .../drm/nouveau/nvkm/subdev/mmu/vmmgp100.c    |   6 +
 include/linux/migrate.h                       |   2 +
 include/linux/migrate_mode.h                  |   1 +
 lib/test_hmm.c                                | 109 ++++++++--
 lib/test_hmm_uapi.h                           |   1 +
 mm/migrate.c                                  |  82 +++++---
 tools/testing/selftests/vm/hmm-tests.c        |  49 +++++
 14 files changed, 524 insertions(+), 101 deletions(-)

-- 
2.20.1

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

             reply index

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-09  1:07 Alistair Popple [this message]
2021-02-09  1:07 ` [Nouveau] [PATCH 1/9] mm/migrate.c: Always allow device private pages to migrate Alistair Popple
2021-02-09 13:39   ` Jason Gunthorpe
2021-02-10  3:40     ` Alistair Popple
2021-02-10 12:56       ` Jason Gunthorpe
2021-02-09  1:07 ` [Nouveau] [PATCH 2/9] mm/migrate.c: Allow pfn flags to be passed to migrate_vma_setup() Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 3/9] mm/migrate: Add a unmap and pin migration mode Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 4/9] Documentation: Add unmap and pin to HMM Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 5/9] hmm-tests: Add test for unmap and pin Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 6/9] nouveau/dmem: Only map migrating pages Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 7/9] nouveau/svm: Refactor nouveau_range_fault Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 8/9] nouveau/dmem: Add support for multiple page types Alistair Popple
2021-02-09  1:07 ` [Nouveau] [PATCH 9/9] nouveau/svm: Implement atomic SVM access Alistair Popple
2021-02-09 10:27 ` [Nouveau] [PATCH 0/9] Add support for SVM atomics in Nouveau Daniel Vetter
2021-02-09 12:57   ` Alistair Popple
2021-02-09 13:35     ` Jason Gunthorpe
2021-02-09 13:39       ` Daniel Vetter
2021-02-09 13:44         ` Jason Gunthorpe
2021-02-09 21:17       ` Jerome Glisse
2021-02-10 17:56         ` Jason Gunthorpe
2021-02-09 13:37     ` Daniel Vetter
2021-02-09 20:53       ` John Hubbard
2021-02-10 12:59         ` Daniel Vetter
2021-02-11  2:26           ` John Hubbard
2021-02-10 17:59         ` Jason Gunthorpe
2021-02-11  7:55           ` Christoph Hellwig
2021-02-17 23:00             ` Alistair Popple

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210209010722.13839-1-apopple@nvidia.com \
    --to=apopple@nvidia.com \
    --cc=akpm@linux-foundation.org \
    --cc=bskeggs@redhat.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=kvm-ppc@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=nouveau@lists.freedesktop.org \
    --cc=rcampbell@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Nouveau Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/nouveau/0 nouveau/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 nouveau nouveau/ https://lore.kernel.org/nouveau \
		nouveau@lists.freedesktop.org
	public-inbox-index nouveau

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.freedesktop.lists.nouveau


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git