* [GIT PULL] Please pull hmm changes
@ 2019-07-30 11:58 Jason Gunthorpe
2019-07-30 20:40 ` pr-tracker-bot
0 siblings, 1 reply; 12+ messages in thread
From: Jason Gunthorpe @ 2019-07-30 11:58 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton
Cc: Christoph Hellwig, dri-devel, linux-mm, David Airlie,
Daniel Vetter, amd-gfx, Kuehling, Felix, Deucher, Alexander,
linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1982 bytes --]
Hi Linus,
Locking fix for nouveau's use of HMM
This small series was posted by Christoph before the merge window, but didn't
make it in time for the PR. It fixes various locking errors in the nouveau
driver's use of the hmm_range_* functions.
The diffstat is a bit big as Christoph did a comprehensive job to move the
obsolete API from the core header and into the driver before fixing its flow,
but the risk of regression from this code motion is low.
I don't intend to often send -rc patches for hmm, but this is entangled with
other changes already, so it is simpler to keep it on the hmm git branch.
Thanks,
Jason
The following changes since commit 5f9e832c137075045d15cd6899ab0505cfb2ca4b:
Linus 5.3-rc1 (2019-07-21 14:05:38 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
for you to fetch changes up to de4ee728465f7c0c29241550e083139b2ce9159c:
nouveau: unlock mmap_sem on all errors from nouveau_range_fault (2019-07-25 16:14:40 -0300)
----------------------------------------------------------------
HMM patches for 5.3-rc
Fix the locking around nouveau's use of the hmm_range_* APIs. It works
correctly in the success case, but many of the the edge cases have missing
unlocks or double unlocks.
----------------------------------------------------------------
Christoph Hellwig (4):
mm/hmm: always return EBUSY for invalid ranges in hmm_range_{fault,snapshot}
mm/hmm: move hmm_vma_range_done and hmm_vma_fault to nouveau
nouveau: remove the block parameter to nouveau_range_fault
nouveau: unlock mmap_sem on all errors from nouveau_range_fault
Documentation/vm/hmm.rst | 2 +-
drivers/gpu/drm/nouveau/nouveau_svm.c | 47 ++++++++++++++++++++++++++++--
include/linux/hmm.h | 54 -----------------------------------
mm/hmm.c | 10 +++----
4 files changed, 49 insertions(+), 64 deletions(-)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull hmm changes
2019-07-30 11:58 [GIT PULL] Please pull hmm changes Jason Gunthorpe
@ 2019-07-30 20:40 ` pr-tracker-bot
0 siblings, 0 replies; 12+ messages in thread
From: pr-tracker-bot @ 2019-07-30 20:40 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Linus Torvalds, Andrew Morton, Christoph Hellwig, dri-devel,
linux-mm, David Airlie, Daniel Vetter, amd-gfx, Kuehling, Felix,
Deucher, Alexander, linux-kernel
The pull request you sent on Tue, 30 Jul 2019 11:58:37 +0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/515f12b9eeed35250d793b7c874707c33f7f6e05
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull hmm changes
2020-08-04 15:06 Jason Gunthorpe
@ 2020-08-05 20:33 ` pr-tracker-bot
0 siblings, 0 replies; 12+ messages in thread
From: pr-tracker-bot @ 2020-08-05 20:33 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Linus Torvalds, linux-kernel, Andrew Morton, linux-mm
The pull request you sent on Tue, 4 Aug 2020 12:06:06 -0300:
> git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/fffe3ae0ee84e25d2befe2ae59bc32aa2b6bc77b
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply [flat|nested] 12+ messages in thread
* [GIT PULL] Please pull hmm changes
@ 2020-08-04 15:06 Jason Gunthorpe
2020-08-05 20:33 ` pr-tracker-bot
0 siblings, 1 reply; 12+ messages in thread
From: Jason Gunthorpe @ 2020-08-04 15:06 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, Andrew Morton, linux-mm
[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]
Hi Linus,
Ralph has been working on nouveau's use of hmm_range_fault() and migrate_vma()
which resulted in this small series of improvements.
There are two expected conflicts. Conflict with the kvm-ppc tree:
https://lore.kernel.org/linux-next/20200730191610.204ed02c@canb.auug.org.au/
This is because the kvm-ppc tree moved __kvmppc_svm_page_out() within the
file. To resolve the diff from 5143192cd410 ("mm/migrate: add a flags
parameter to migrate_vma") needs to be applied to the new location:
- mig.src_owner = &kvmppc_uvmem_pgmap;
++ mig.pgmap_owner = &kvmppc_uvmem_pgmap;
++ mig.flags = MIGRATE_VMA_SELECT_DEVICE_PRIVATE;
There is an accidental conflict with the DRM tree, due to a confusion the same
patch got applied to both trees and we couldn't undo it in time. Please take
the hmm version of the resolution.
Since neither DRM or kvm-ppc have been merged by you, I've made a very hacky
for-linus-hmm-merged tag with my resolutions for your reference.
Thanks,
Jason
The following changes since commit dcb7fd82c75ee2d6e6f9d8cc71c52519ed52e258:
Linux 5.8-rc4 (2020-07-05 16:20:22 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
for you to fetch changes up to 7d17e83abec1be3355260b3e4812044c65c32907:
mm/hmm/test: use the new migration invalidation (2020-07-28 16:20:33 -0300)
----------------------------------------------------------------
hmm related patches for 5.9
This series adds reporting of the page table order from hmm_range_fault()
and some optimization of migrate_vma():
- Report the size of the page table mapping out of hmm_range_fault(). This
makes it easier to establish a large/huge/etc mapping in the device's
page table.
- Allow devices to ignore the invalidations during migration in cases
where the migration is not going to change pages. For instance migrating
pages to a device does not require the device to invalidate pages
already in the device.
- Update nouveau and hmm_tests to use the above
----------------------------------------------------------------
Ralph Campbell (10):
mm/hmm: provide the page mapping order in hmm_range_fault()
mm/hmm: add tests for hmm_pfn_to_map_order()
nouveau/hmm: fault one page at a time
nouveau: fix mapping 2MB sysmem pages
nouveau/hmm: support mapping large sysmem pages
nouveau: fix storing invalid ptes
mm/migrate: add a flags parameter to migrate_vma
mm/notifier: add migration invalidation type
nouveau/svm: use the new migration invalidation
mm/hmm/test: use the new migration invalidation
arch/powerpc/kvm/book3s_hv_uvmem.c | 4 +-
drivers/gpu/drm/nouveau/nouveau_dmem.c | 19 +-
drivers/gpu/drm/nouveau/nouveau_svm.c | 257 +++++++++------------
drivers/gpu/drm/nouveau/nouveau_svm.h | 13 +-
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c | 5 +-
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c | 91 +++++++-
include/linux/hmm.h | 24 +-
include/linux/migrate.h | 16 +-
include/linux/mmu_notifier.h | 6 +
lib/test_hmm.c | 47 ++--
lib/test_hmm_uapi.h | 4 +
mm/hmm.c | 16 +-
mm/migrate.c | 14 +-
tools/testing/selftests/vm/hmm-tests.c | 94 +++++++-
14 files changed, 410 insertions(+), 200 deletions(-)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull hmm changes
2020-06-01 18:25 Jason Gunthorpe
@ 2020-06-02 22:20 ` pr-tracker-bot
0 siblings, 0 replies; 12+ messages in thread
From: pr-tracker-bot @ 2020-06-02 22:20 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Linus Torvalds, linux-kernel, Andrew Morton, linux-mm,
linux-rdma, linux-kernel
The pull request you sent on Mon, 1 Jun 2020 15:25:52 -0300:
> git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/cfa3b8068b09f25037146bfd5eed041b78878bee
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 12+ messages in thread
* [GIT PULL] Please pull hmm changes
@ 2020-06-01 18:25 Jason Gunthorpe
2020-06-02 22:20 ` pr-tracker-bot
0 siblings, 1 reply; 12+ messages in thread
From: Jason Gunthorpe @ 2020-06-01 18:25 UTC (permalink / raw)
To: Linus Torvalds, linux-kernel, Andrew Morton, linux-mm
Cc: linux-rdma, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2698 bytes --]
Hi Linus,
This is a small update for 5.8 mainly including the selftest from
Ralph and finishing the hmm_range_fault() simplification started in
the last merge window.
Regards,
Jason
The following changes since commit 0e698dfa282211e414076f9dc7e83c1c288314fd:
Linux 5.7-rc4 (2020-05-03 14:56:04 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
for you to fetch changes up to f07e2f6be37a750737b93f5635485171ad459eb9:
MAINTAINERS: add HMM selftests (2020-05-19 16:48:31 -0300)
----------------------------------------------------------------
hmm related patches for 5.8
This series adds a selftest for hmm_range_fault() and several of the
DEVICE_PRIVATE migration related actions, and another simplification for
hmm_range_fault()'s API.
- Simplify hmm_range_fault() with a simpler return code, no
HMM_PFN_SPECIAL, and no customizable output PFN format
- Add a selftest for hmm_range_fault() and DEVICE_PRIVATE related
functionality
----------------------------------------------------------------
Jason Gunthorpe (4):
mm/hmm: make hmm_range_fault return 0 or -1
drm/amdgpu: remove dead code after hmm_range_fault()
mm/hmm: remove HMM_PFN_SPECIAL
mm/hmm: remove the customizable pfn format from hmm_range_fault
Ralph Campbell (3):
mm/hmm/test: add selftest driver for HMM
mm/hmm/test: add selftests for HMM
MAINTAINERS: add HMM selftests
Documentation/vm/hmm.rst | 30 +-
MAINTAINERS | 2 +
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 56 +-
drivers/gpu/drm/nouveau/nouveau_dmem.c | 27 +-
drivers/gpu/drm/nouveau/nouveau_dmem.h | 3 +-
drivers/gpu/drm/nouveau/nouveau_svm.c | 94 ++-
include/linux/hmm.h | 111 +--
lib/Kconfig.debug | 13 +
lib/Makefile | 1 +
lib/test_hmm.c | 1164 ++++++++++++++++++++++++++
lib/test_hmm_uapi.h | 59 ++
mm/hmm.c | 185 ++---
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +
tools/testing/selftests/vm/config | 2 +
tools/testing/selftests/vm/hmm-tests.c | 1359 +++++++++++++++++++++++++++++++
tools/testing/selftests/vm/run_vmtests | 16 +
tools/testing/selftests/vm/test_hmm.sh | 97 +++
18 files changed, 2934 insertions(+), 289 deletions(-)
create mode 100644 lib/test_hmm.c
create mode 100644 lib/test_hmm_uapi.h
create mode 100644 tools/testing/selftests/vm/hmm-tests.c
create mode 100755 tools/testing/selftests/vm/test_hmm.sh
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull HMM changes
2020-03-30 19:54 ` Nathan Chancellor
@ 2020-03-30 20:08 ` Jason Gunthorpe
0 siblings, 0 replies; 12+ messages in thread
From: Jason Gunthorpe @ 2020-03-30 20:08 UTC (permalink / raw)
To: Nathan Chancellor; +Cc: Linus Torvalds, linux-kernel, Andrew Morton, linux-mm
On Mon, Mar 30, 2020 at 12:54:03PM -0700, Nathan Chancellor wrote:
> > Jason Gunthorpe (17):
> > mm/hmm: add missing unmaps of the ptep during hmm_vma_handle_pte()
> > mm/hmm: do not call hmm_vma_walk_hole() while holding a spinlock
> > mm/hmm: add missing pfns set to hmm_vma_walk_pmd()
> > mm/hmm: add missing call to hmm_range_need_fault() before returning EFAULT
> > mm/hmm: reorganize how !pte_present is handled in hmm_vma_handle_pte()
> > mm/hmm: return -EFAULT when setting HMM_PFN_ERROR on requested valid pages
> > mm/hmm: add missing call to hmm_pte_need_fault in HMM_PFN_SPECIAL handling
> > mm/hmm: do not check pmd_protnone twice in hmm_vma_handle_pmd()
> > mm/hmm: remove pgmap checking for devmap pages
> > mm/hmm: return the fault type from hmm_pte_need_fault()
> > mm/hmm: remove unused code and tidy comments
> > mm/hmm: remove HMM_FAULT_SNAPSHOT
> > mm/hmm: remove the CONFIG_TRANSPARENT_HUGEPAGE #ifdef
>
> This patch causes an error on arm32 all{mod,yes}config because pmd_pfn
> is only defined when CONFIG_ARM_LPAE is set, which is a dependency of
> CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE and CONFIG_TRANSPARENT_HUGEPAGE.
Oh! I'm again surprised 0-day did not catch on to this. linux-next
surely would have, but things got there later than I would have
preferred due to the world being upside down right now :(
> https://elixir.bootlin.com/linux/v5.6/source/arch/arm/include/asm/pgtable-3level.h#L236
> https://elixir.bootlin.com/linux/v5.6/source/arch/arm/include/asm/pgtable.h#L29
> https://elixir.bootlin.com/linux/v5.6/source/arch/arm/Kconfig#L1579
>
> No idea how to rectify that but thought I would let you know.
I'll just drop the patch. Next cycle we can add some comment here as
this requirement is a hard to notice.
The for-linus-hmm tag is updated now to reflect this.
Thanks a lot,
Jason
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull HMM changes
2020-03-30 17:57 [GIT PULL] Please pull HMM changes Jason Gunthorpe
@ 2020-03-30 19:54 ` Nathan Chancellor
2020-03-30 20:08 ` Jason Gunthorpe
0 siblings, 1 reply; 12+ messages in thread
From: Nathan Chancellor @ 2020-03-30 19:54 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: Linus Torvalds, linux-kernel, Andrew Morton, linux-mm
On Mon, Mar 30, 2020 at 02:57:48PM -0300, Jason Gunthorpe wrote:
> Hi Linus,
>
> This series arose from a review of hmm_range_fault() by Christoph, Ralph and
> myself. Several bug fixes and some general clarity.
>
> hmm_range_fault() is being used by these 'SVM' style drivers to
> non-destructively read the page tables. It is very similar to get_user_pages()
> except that the output is an array of PFNs and per-pfn flags, and it has
> various modes of reading.
>
> This is necessary before RDMA ODP can be converted, as we don't want to have
> weird corner case regressions, which is still a looking forward item. Ralph
> has a nice tester for this routine, but it is waiting for feedback from the
> selftests maintainers.
>
> Regards,
> Jason
>
> The following changes since commit f8788d86ab28f61f7b46eb6be375f8a726783636:
>
> Linux 5.6-rc3 (2020-02-23 16:17:42 -0800)
>
> are available in the Git repository at:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
>
> for you to fetch changes up to 9cee0e8c6f1eb4b5e56d3eb7f5d47b05637bab4f:
>
> mm/hmm: return error for non-vma snapshots (2020-03-27 20:19:25 -0300)
>
> ----------------------------------------------------------------
> hmm related patches for 5.7
>
> This series focuses on corner case bug fixes and general clarity
> improvements to hmm_range_fault().
>
> - 9 bug fixes
>
> - Allow pgmap to track the 'owner' of a DEVICE_PRIVATE - in this case the
> owner tells the driver if it can understand the DEVICE_PRIVATE page or
> not. Use this to resolve a bug in nouveau where it could touch
> DEVICE_PRIVATE pages from other drivers.
>
> - Remove a bunch of dead, redundant or unused code and flags
>
> - Clarity improvements to hmm_range_fault()
>
> ----------------------------------------------------------------
> Christoph Hellwig (9):
> mm/hmm: don't provide a stub for hmm_range_fault()
> mm/hmm: remove the unused HMM_FAULT_ALLOW_RETRY flag
> mm/hmm: simplify hmm_vma_walk_hugetlb_entry()
> mm/hmm: don't handle the non-fault case in hmm_vma_walk_hole_()
> mm: merge hmm_vma_do_fault into into hmm_vma_walk_hole_
> memremap: add an owner field to struct dev_pagemap
> mm: handle multiple owners of device private pages in migrate_vma
> mm: simplify device private page handling in hmm_range_fault
> mm/hmm: check the device private page owner in hmm_range_fault()
>
> Jason Gunthorpe (17):
> mm/hmm: add missing unmaps of the ptep during hmm_vma_handle_pte()
> mm/hmm: do not call hmm_vma_walk_hole() while holding a spinlock
> mm/hmm: add missing pfns set to hmm_vma_walk_pmd()
> mm/hmm: add missing call to hmm_range_need_fault() before returning EFAULT
> mm/hmm: reorganize how !pte_present is handled in hmm_vma_handle_pte()
> mm/hmm: return -EFAULT when setting HMM_PFN_ERROR on requested valid pages
> mm/hmm: add missing call to hmm_pte_need_fault in HMM_PFN_SPECIAL handling
> mm/hmm: do not check pmd_protnone twice in hmm_vma_handle_pmd()
> mm/hmm: remove pgmap checking for devmap pages
> mm/hmm: return the fault type from hmm_pte_need_fault()
> mm/hmm: remove unused code and tidy comments
> mm/hmm: remove HMM_FAULT_SNAPSHOT
> mm/hmm: remove the CONFIG_TRANSPARENT_HUGEPAGE #ifdef
This patch causes an error on arm32 all{mod,yes}config because pmd_pfn
is only defined when CONFIG_ARM_LPAE is set, which is a dependency of
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE and CONFIG_TRANSPARENT_HUGEPAGE.
$ make -j$(nproc) -s ARCH=arm CC=clang CROSS_COMPILE=arm-linux-gnueabi- O=out/arm32 distclean allyesconfig mm/hmm.o
mm/hmm.c:207:8: error: implicit declaration of function 'pmd_pfn'
[-Werror,-Wimplicit-function-declaration]
pfn = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT);
^
1 error generated.
https://elixir.bootlin.com/linux/v5.6/source/arch/arm/include/asm/pgtable-3level.h#L236
https://elixir.bootlin.com/linux/v5.6/source/arch/arm/include/asm/pgtable.h#L29
https://elixir.bootlin.com/linux/v5.6/source/arch/arm/Kconfig#L1579
No idea how to rectify that but thought I would let you know.
Cheers,
Nathan
^ permalink raw reply [flat|nested] 12+ messages in thread
* [GIT PULL] Please pull HMM changes
@ 2020-03-30 17:57 Jason Gunthorpe
2020-03-30 19:54 ` Nathan Chancellor
0 siblings, 1 reply; 12+ messages in thread
From: Jason Gunthorpe @ 2020-03-30 17:57 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-kernel, Andrew Morton, linux-mm
[-- Attachment #1: Type: text/plain, Size: 4017 bytes --]
Hi Linus,
This series arose from a review of hmm_range_fault() by Christoph, Ralph and
myself. Several bug fixes and some general clarity.
hmm_range_fault() is being used by these 'SVM' style drivers to
non-destructively read the page tables. It is very similar to get_user_pages()
except that the output is an array of PFNs and per-pfn flags, and it has
various modes of reading.
This is necessary before RDMA ODP can be converted, as we don't want to have
weird corner case regressions, which is still a looking forward item. Ralph
has a nice tester for this routine, but it is waiting for feedback from the
selftests maintainers.
Regards,
Jason
The following changes since commit f8788d86ab28f61f7b46eb6be375f8a726783636:
Linux 5.6-rc3 (2020-02-23 16:17:42 -0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
for you to fetch changes up to 9cee0e8c6f1eb4b5e56d3eb7f5d47b05637bab4f:
mm/hmm: return error for non-vma snapshots (2020-03-27 20:19:25 -0300)
----------------------------------------------------------------
hmm related patches for 5.7
This series focuses on corner case bug fixes and general clarity
improvements to hmm_range_fault().
- 9 bug fixes
- Allow pgmap to track the 'owner' of a DEVICE_PRIVATE - in this case the
owner tells the driver if it can understand the DEVICE_PRIVATE page or
not. Use this to resolve a bug in nouveau where it could touch
DEVICE_PRIVATE pages from other drivers.
- Remove a bunch of dead, redundant or unused code and flags
- Clarity improvements to hmm_range_fault()
----------------------------------------------------------------
Christoph Hellwig (9):
mm/hmm: don't provide a stub for hmm_range_fault()
mm/hmm: remove the unused HMM_FAULT_ALLOW_RETRY flag
mm/hmm: simplify hmm_vma_walk_hugetlb_entry()
mm/hmm: don't handle the non-fault case in hmm_vma_walk_hole_()
mm: merge hmm_vma_do_fault into into hmm_vma_walk_hole_
memremap: add an owner field to struct dev_pagemap
mm: handle multiple owners of device private pages in migrate_vma
mm: simplify device private page handling in hmm_range_fault
mm/hmm: check the device private page owner in hmm_range_fault()
Jason Gunthorpe (17):
mm/hmm: add missing unmaps of the ptep during hmm_vma_handle_pte()
mm/hmm: do not call hmm_vma_walk_hole() while holding a spinlock
mm/hmm: add missing pfns set to hmm_vma_walk_pmd()
mm/hmm: add missing call to hmm_range_need_fault() before returning EFAULT
mm/hmm: reorganize how !pte_present is handled in hmm_vma_handle_pte()
mm/hmm: return -EFAULT when setting HMM_PFN_ERROR on requested valid pages
mm/hmm: add missing call to hmm_pte_need_fault in HMM_PFN_SPECIAL handling
mm/hmm: do not check pmd_protnone twice in hmm_vma_handle_pmd()
mm/hmm: remove pgmap checking for devmap pages
mm/hmm: return the fault type from hmm_pte_need_fault()
mm/hmm: remove unused code and tidy comments
mm/hmm: remove HMM_FAULT_SNAPSHOT
mm/hmm: remove the CONFIG_TRANSPARENT_HUGEPAGE #ifdef
mm/hmm: use device_private_entry_to_pfn()
mm/hmm: do not unconditionally set pfns when returning EBUSY
mm/hmm: do not set pfns when returning an error code
mm/hmm: return error for non-vma snapshots
Documentation/vm/hmm.rst | 12 +-
arch/powerpc/kvm/book3s_hv_uvmem.c | 3 +
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 +-
drivers/gpu/drm/nouveau/nouveau_dmem.c | 19 +-
drivers/gpu/drm/nouveau/nouveau_svm.c | 3 +-
include/linux/hmm.h | 125 +--------
include/linux/memremap.h | 4 +
include/linux/migrate.h | 8 +
mm/hmm.c | 476 +++++++++++++-------------------
mm/memremap.c | 4 +
mm/migrate.c | 9 +-
11 files changed, 227 insertions(+), 439 deletions(-)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull hmm changes
2019-07-09 19:24 [GIT PULL] Please pull hmm changes Jason Gunthorpe
2019-07-15 2:48 ` Linus Torvalds
@ 2019-07-15 3:00 ` pr-tracker-bot
1 sibling, 0 replies; 12+ messages in thread
From: pr-tracker-bot @ 2019-07-15 3:00 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Linus Torvalds, Andrew Morton, Dan Williams, Christoph Hellwig,
dri-devel, linux-mm, David Airlie, Daniel Vetter, amd-gfx,
Kuehling, Felix, Deucher, Alexander, linux-rdma, linux-kernel
The pull request you sent on Tue, 9 Jul 2019 19:24:21 +0000:
> git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/fec88ab0af9706b2201e5daf377c5031c62d11f7
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [GIT PULL] Please pull hmm changes
2019-07-09 19:24 [GIT PULL] Please pull hmm changes Jason Gunthorpe
@ 2019-07-15 2:48 ` Linus Torvalds
2019-07-15 3:00 ` pr-tracker-bot
1 sibling, 0 replies; 12+ messages in thread
From: Linus Torvalds @ 2019-07-15 2:48 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Andrew Morton, Dan Williams, Christoph Hellwig, dri-devel,
linux-mm, David Airlie, Daniel Vetter, amd-gfx, Kuehling, Felix,
Deucher, Alexander, linux-rdma, linux-kernel
On Tue, Jul 9, 2019 at 12:24 PM Jason Gunthorpe <jgg@mellanox.com> wrote:
>
> I'm sending it early as it is now a dependency for several patches in
> mm's quilt.
.. but I waited to merge it until I had time to review it more
closely, because I expected the review to be painful.
I'm happy to say that I was overly pessimistic, and that instead of
finding things to hate, I found it all looking good.
Particularly the whole "use reference counts properly, so that
lifetimes make sense and all those nasty cases can't happen" parts.
It's all merged, just waiting for the test-build to verify that I
didn't miss anything (well, at least nothing obvious).
Linus
^ permalink raw reply [flat|nested] 12+ messages in thread
* [GIT PULL] Please pull hmm changes
@ 2019-07-09 19:24 Jason Gunthorpe
2019-07-15 2:48 ` Linus Torvalds
2019-07-15 3:00 ` pr-tracker-bot
0 siblings, 2 replies; 12+ messages in thread
From: Jason Gunthorpe @ 2019-07-09 19:24 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton
Cc: Dan Williams, Christoph Hellwig, dri-devel, linux-mm,
David Airlie, Daniel Vetter, amd-gfx, Kuehling, Felix, Deucher,
Alexander, linux-rdma, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 9266 bytes --]
Hi Linus,
As was discussed some time ago here are the mostly -mm patches related
to hmm functions. In agreement with Andrew we split this out from
quilt into a git topic branch so it can be shared between the DRM and
RDMA git trees. However, this cycle did not see dependencies with work
in DRM or RDMA that required a topic merge. I expect that work will
start to get ready next cycle and we will see a need for a cross-tree
topic merge then.
I'm sending it early as it is now a dependency for several patches in
mm's quilt.
This has been an exciting topic branch for conflicts, you'll need the
below simple resolution in the merge commit to make it compile
(lockdep_assert_held_exclusive() was renamed to
lockdep_assert_held_write())
Otherwise, for reference to all parties, here is how the conflicts were
handled:
- Several small patches from -mm quilt were moved to this tree to simplify
conflict management, only Ira's 'fix release_pages()' patch was not hmm
related.
- DRM introduced a new users of the hmm_range_register() API. We worked
with AMDGPU to ensure that their new user could use the revised API via
the below trivial merge fixup with DRM:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -783,7 +783,7 @@ int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages)
0 : range->flags[HMM_PFN_WRITE];
range->pfn_flags_mask = 0;
range->pfns = pfns;
- hmm_range_register(range, mm, start,
+ hmm_range_register(range, mirror, start,
start + ttm->num_pages * PAGE_SIZE, PAGE_SHIFT);
retry:
- ARM64 has a patch series going through -mm with a trivial
conflict ("Devmap cleanups + arm64 support"), Andrew has re-applied this
in quilt onto linux-next and will send it
- The memreap sub-section changes in -mm has 5 hunk conflict with the
memremap changes here. Andrew reapplied Dan's series ontop of
Christoph's series in linux-next and will send it.
The tag for-linus-hmm-merged with my merge resolution to your tree is
also available to pull.
Thanks,
Jason
diff --cc mm/hmm.c
index d48b9283725a90,f702a3895d05d8..e1eedef129cf5c
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@@ -42,16 -54,11 +42,16 @@@ static const struct mmu_notifier_ops hm
*/
static struct hmm *hmm_get_or_create(struct mm_struct *mm)
{
- struct hmm *hmm = mm_get_hmm(mm);
- bool cleanup = false;
+ struct hmm *hmm;
- lockdep_assert_held_exclusive(&mm->mmap_sem);
- if (hmm)
- return hmm;
++ lockdep_assert_held_write(&mm->mmap_sem);
+
+ /* Abuse the page_table_lock to also protect mm->hmm. */
+ spin_lock(&mm->page_table_lock);
+ hmm = mm->hmm;
+ if (mm->hmm && kref_get_unless_zero(&mm->hmm->kref))
+ goto out_unlock;
+ spin_unlock(&mm->page_table_lock);
hmm = kmalloc(sizeof(*hmm), GFP_KERNEL);
if (!hmm)
@@@ -245,8 -277,8 +245,8 @@@ static const struct mmu_notifier_ops hm
*/
int hmm_mirror_register(struct hmm_mirror *mirror, struct mm_struct *mm)
{
- lockdep_assert_held_exclusive(&mm->mmap_sem);
++ lockdep_assert_held_write(&mm->mmap_sem);
+
/* Sanity check */
if (!mm || !mirror || !mirror->ops)
return -EINVAL;
The following changes since commit 6fbc7275c7a9ba97877050335f290341a1fd8dbf:
Linux 5.2-rc7 (2019-06-30 11:25:36 +0800)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git tags/for-linus-hmm
for you to fetch changes up to cc5dfd59e375f4d0f2b64643723d16b38b2f2d78:
Merge branch 'hmm-devmem-cleanup.4' into rdma.git hmm (2019-07-02 15:10:45 -0300)
----------------------------------------------------------------
HMM patches for 5.3
Improvements and bug fixes for the hmm interface in the kernel:
- Improve clarity, locking and APIs related to the 'hmm mirror' feature
merged last cycle. In linux-next we now see AMDGPU and nouveau to be
using this API.
- Remove old or transitional hmm APIs. These are hold overs from the past
with no users, or APIs that existed only to manage cross tree conflicts.
There are still a few more of these cleanups that didn't make the merge
window cut off.
- Improve some core mm APIs:
* export alloc_pages_vma() for driver use
* refactor into devm_request_free_mem_region() to manage
DEVICE_PRIVATE resource reservations
* refactor duplicative driver code into the core dev_pagemap
struct
- Remove hmm wrappers of improved core mm APIs, instead have drivers use
the simplified API directly
- Remove DEVICE_PUBLIC
- Simplify the kconfig flow for the hmm users and core code
----------------------------------------------------------------
Christoph Hellwig (24):
mm: remove the unused ARCH_HAS_HMM_DEVICE Kconfig option
mm: remove the struct hmm_device infrastructure
mm: remove MEMORY_DEVICE_PUBLIC support
mm: don't clear ->mapping in hmm_devmem_free
mm: export alloc_pages_vma
mm: factor out a devm_request_free_mem_region helper
memremap: validate the pagemap type passed to devm_memremap_pages
memremap: move dev_pagemap callbacks into a separate structure
memremap: pass a struct dev_pagemap to ->kill and ->cleanup
memremap: lift the devmap_enable manipulation into devm_memremap_pages
memremap: add a migrate_to_ram method to struct dev_pagemap_ops
memremap: remove the data field in struct dev_pagemap
memremap: replace the altmap_valid field with a PGMAP_ALTMAP_VALID flag
memremap: provide an optional internal refcount in struct dev_pagemap
device-dax: use the dev_pagemap internal refcount
PCI/P2PDMA: use the dev_pagemap internal refcount
nouveau: use alloc_page_vma directly
nouveau: use devm_memremap_pages directly
mm: remove hmm_vma_alloc_locked_page
mm: remove hmm_devmem_add
mm: simplify ZONE_DEVICE page private data
mm: sort out the DEVICE_PRIVATE Kconfig mess
mm: remove the HMM config option
mm: don't select MIGRATE_VMA_HELPER from HMM_MIRROR
Ira Weiny (1):
mm/swap: fix release_pages() when releasing devmap pages
Jason Gunthorpe (15):
mm/hmm.c: suppress compilation warnings when CONFIG_HUGETLB_PAGE is not set
mm/hmm: fix use after free with struct hmm in the mmu notifiers
mm/hmm: Use hmm_mirror not mm as an argument for hmm_range_register
mm/hmm: Hold a mmgrab from hmm to mm
mm/hmm: Simplify hmm_get_or_create and make it reliable
mm/hmm: Remove duplicate condition test before wait_event_timeout
mm/hmm: Do not use list*_rcu() for hmm->ranges
mm/hmm: Hold on to the mmget for the lifetime of the range
mm/hmm: Use lockdep instead of comments
mm/hmm: Remove racy protection against double-unregistration
mm/hmm: Poison hmm_range during unregister
mm/hmm: Remove confusing comment and logic from hmm_release
mm/hmm: Fix error flows in hmm_invalidate_range_start
Merge tag 'v5.2-rc7' into rdma.git hmm
Merge branch 'hmm-devmem-cleanup.4' into rdma.git hmm
Kuehling, Felix (1):
mm/hmm: Only set FAULT_FLAG_ALLOW_RETRY for non-blocking
Philip Yang (1):
mm/hmm: support automatic NUMA balancing
Ralph Campbell (2):
mm/hmm: update HMM documentation
mm/hmm: clean up some coding style and comments
Documentation/vm/hmm.rst | 166 ++++------
arch/powerpc/mm/mem.c | 10 +-
arch/x86/mm/init_64.c | 8 +-
drivers/dax/dax-private.h | 4 -
drivers/dax/device.c | 41 +--
drivers/dax/pmem/core.c | 2 +-
drivers/gpu/drm/nouveau/Kconfig | 6 +-
drivers/gpu/drm/nouveau/nouveau_dmem.c | 103 +++---
drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +-
drivers/nvdimm/pfn_devs.c | 3 +-
drivers/nvdimm/pmem.c | 51 ++-
drivers/pci/p2pdma.c | 52 +--
fs/proc/task_mmu.c | 2 +-
include/linux/hmm.h | 302 ++---------------
include/linux/ioport.h | 3 +-
include/linux/memremap.h | 75 +++--
include/linux/mm.h | 28 +-
include/linux/mm_types.h | 4 +-
include/linux/swapops.h | 15 -
kernel/fork.c | 1 -
kernel/memremap.c | 194 ++++++-----
kernel/resource.c | 39 +++
mm/Kconfig | 50 +--
mm/Makefile | 2 +-
mm/gup.c | 7 -
mm/hmm.c | 587 ++++++++-------------------------
mm/madvise.c | 2 +-
mm/memcontrol.c | 13 +-
mm/memory-failure.c | 6 +-
mm/memory.c | 49 +--
mm/memory_hotplug.c | 6 +-
mm/mempolicy.c | 1 +
mm/migrate.c | 28 +-
mm/page_alloc.c | 13 +-
mm/swap.c | 13 +-
tools/testing/nvdimm/test/iomap.c | 57 +++-
36 files changed, 619 insertions(+), 1326 deletions(-)
(diffstat from tag for-linus-hmm-merged)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2020-08-05 20:33 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-30 11:58 [GIT PULL] Please pull hmm changes Jason Gunthorpe
2019-07-30 20:40 ` pr-tracker-bot
-- strict thread matches above, loose matches on Subject: below --
2020-08-04 15:06 Jason Gunthorpe
2020-08-05 20:33 ` pr-tracker-bot
2020-06-01 18:25 Jason Gunthorpe
2020-06-02 22:20 ` pr-tracker-bot
2020-03-30 17:57 [GIT PULL] Please pull HMM changes Jason Gunthorpe
2020-03-30 19:54 ` Nathan Chancellor
2020-03-30 20:08 ` Jason Gunthorpe
2019-07-09 19:24 [GIT PULL] Please pull hmm changes Jason Gunthorpe
2019-07-15 2:48 ` Linus Torvalds
2019-07-15 3:00 ` pr-tracker-bot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).