Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 00/19] pin_user_pages(): reduced-risk series for Linux 5.5
@ 2019-11-25  4:19 John Hubbard
  2019-11-25  4:19 ` [PATCH 01/19] mm/gup: factor out duplicate code from four routines John Hubbard
                   ` (18 more replies)
  0 siblings, 19 replies; 25+ messages in thread
From: John Hubbard @ 2019-11-25  4:19 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Al Viro, Alex Williamson, Benjamin Herrenschmidt,
	Björn Töpel, Christoph Hellwig, Dan Williams,
	Daniel Vetter, Dave Chinner, David Airlie, David S . Miller,
	Ira Weiny, Jan Kara, Jason Gunthorpe, Jens Axboe,
	Jonathan Corbet, Jérôme Glisse, Magnus Karlsson,
	Mauro Carvalho Chehab, Michael Ellerman, Michal Hocko,
	Mike Kravetz, Paul Mackerras, Shuah Khan, Vlastimil Babka, bpf,
	dri-devel, kvm, linux-block, linux-doc, linux-fsdevel,
	linux-kselftest, linux-media, linux-rdma, linuxppc-dev, netdev,
	linux-mm, LKML, John Hubbard

Hi,

Here is a set of well-reviewed (expect for one patch), lower-risk  items
that can go into Linux 5.5. The one patch that wasn't reviewed is the
powerpc conversion, and it's still at this point a no-op, because
tracking isn't yet activated.

This is based on linux-next: b9d3d01405061bb42358fe53f824e894a1922ced
("Add linux-next specific files for 20191122").

This is essentially a cut-down v8 of "mm/gup: track dma-pinned pages:
FOLL_PIN" [1], and with one of the VFIO patches split into two patches.
The idea here is to get this long list of "noise" checked into 5.5, so
that the actual, higher-risk "track FOLL_PIN pages" (which is deferred:
not part of this series) will be a much shorter patchset to review.

For the v4l2-core changes, I've left those here (instead of sending
them separately to the -media tree), in order to get the name change
done now (put_user_page --> unpin_user_page). However, I've added a Cc
stable, as recommended during the last round of reviews.

Here are the relevant notes from the original cover letter, edited to
match the current situation:

This is a prerequisite to tracking dma-pinned pages. That in turn is a
prerequisite to solving the larger problem of proper interactions
between file-backed pages, and [R]DMA activities, as discussed in [1],
[2], [3], and in a remarkable number of email threads since about
2017. :)

A new internal gup flag, FOLL_PIN is introduced, and thoroughly
documented in the last patch's Documentation/vm/pin_user_pages.rst.

I believe that this will provide a good starting point for doing the
layout lease work that Ira Weiny has been working on. That's because
these new wrapper functions provide a clean, constrained, systematically
named set of functionality that, again, is required in order to even
know if a page is "dma-pinned".

In contrast to earlier approaches, the page tracking can be
incrementally applied to the kernel call sites that, until now, have
been simply calling get_user_pages() ("gup"). In other words, opt-in by
changing from this:

    get_user_pages() (sets FOLL_GET)
    put_page()

to this:
    pin_user_pages() (sets FOLL_PIN)
    put_user_page()

Because there are interdependencies with FOLL_LONGTERM, a similar
conversion as for FOLL_PIN, was applied. The change was from this:

    get_user_pages(FOLL_LONGTERM) (also sets FOLL_GET)
    put_page()

to this:
    pin_longterm_pages() (sets FOLL_PIN | FOLL_LONGTERM)
    put_user_page()

[1] https://lore.kernel.org/r/20191121071354.456618-1-jhubbard@nvidia.com

thanks,
John Hubbard
NVIDIA


Dan Williams (1):
  mm: Cleanup __put_devmap_managed_page() vs ->page_free()

John Hubbard (18):
  mm/gup: factor out duplicate code from four routines
  mm/gup: move try_get_compound_head() to top, fix minor issues
  goldish_pipe: rename local pin_user_pages() routine
  mm: fix get_user_pages_remote()'s handling of FOLL_LONGTERM
  vfio: fix FOLL_LONGTERM use, simplify get_user_pages_remote() call
  mm/gup: introduce pin_user_pages*() and FOLL_PIN
  goldish_pipe: convert to pin_user_pages() and put_user_page()
  IB/{core,hw,umem}: set FOLL_PIN via pin_user_pages*(), fix up ODP
  mm/process_vm_access: set FOLL_PIN via pin_user_pages_remote()
  drm/via: set FOLL_PIN via pin_user_pages_fast()
  fs/io_uring: set FOLL_PIN via pin_user_pages()
  net/xdp: set FOLL_PIN via pin_user_pages()
  media/v4l2-core: set pages dirty upon releasing DMA buffers
  media/v4l2-core: pin_user_pages (FOLL_PIN) and put_user_page()
    conversion
  vfio, mm: pin_user_pages (FOLL_PIN) and put_user_page() conversion
  powerpc: book3s64: convert to pin_user_pages() and put_user_page()
  mm/gup_benchmark: use proper FOLL_WRITE flags instead of hard-coding
    "1"
  mm, tree-wide: rename put_user_page*() to unpin_user_page*()

 Documentation/core-api/index.rst            |   1 +
 Documentation/core-api/pin_user_pages.rst   | 233 ++++++++++++++
 arch/powerpc/mm/book3s64/iommu_api.c        |  12 +-
 drivers/gpu/drm/via/via_dmablit.c           |   6 +-
 drivers/infiniband/core/umem.c              |   4 +-
 drivers/infiniband/core/umem_odp.c          |  13 +-
 drivers/infiniband/hw/hfi1/user_pages.c     |   4 +-
 drivers/infiniband/hw/mthca/mthca_memfree.c |   8 +-
 drivers/infiniband/hw/qib/qib_user_pages.c  |   4 +-
 drivers/infiniband/hw/qib/qib_user_sdma.c   |   8 +-
 drivers/infiniband/hw/usnic/usnic_uiom.c    |   4 +-
 drivers/infiniband/sw/siw/siw_mem.c         |   4 +-
 drivers/media/v4l2-core/videobuf-dma-sg.c   |   8 +-
 drivers/nvdimm/pmem.c                       |   6 -
 drivers/platform/goldfish/goldfish_pipe.c   |  35 +--
 drivers/vfio/vfio_iommu_type1.c             |  35 +--
 fs/io_uring.c                               |   6 +-
 include/linux/mm.h                          |  77 +++--
 mm/gup.c                                    | 332 +++++++++++++-------
 mm/gup_benchmark.c                          |   9 +-
 mm/memremap.c                               |  80 ++---
 mm/process_vm_access.c                      |  28 +-
 net/xdp/xdp_umem.c                          |   4 +-
 tools/testing/selftests/vm/gup_benchmark.c  |   6 +-
 24 files changed, 642 insertions(+), 285 deletions(-)
 create mode 100644 Documentation/core-api/pin_user_pages.rst

-- 
2.24.0


^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, back to index

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-25  4:19 [PATCH 00/19] pin_user_pages(): reduced-risk series for Linux 5.5 John Hubbard
2019-11-25  4:19 ` [PATCH 01/19] mm/gup: factor out duplicate code from four routines John Hubbard
2019-11-25  4:19 ` [PATCH 02/19] mm/gup: move try_get_compound_head() to top, fix minor issues John Hubbard
2019-11-25  4:19 ` [PATCH 03/19] mm: Cleanup __put_devmap_managed_page() vs ->page_free() John Hubbard
2019-11-25  4:19 ` [PATCH 04/19] goldish_pipe: rename local pin_user_pages() routine John Hubbard
2019-11-25  4:19 ` [PATCH 05/19] mm: fix get_user_pages_remote()'s handling of FOLL_LONGTERM John Hubbard
2019-11-25  4:19 ` [PATCH 06/19] vfio: fix FOLL_LONGTERM use, simplify get_user_pages_remote() call John Hubbard
2019-11-25  4:19 ` [PATCH 07/19] mm/gup: introduce pin_user_pages*() and FOLL_PIN John Hubbard
2019-11-25  7:47   ` kbuild test robot
2019-11-25  8:44   ` kbuild test robot
2019-11-25 20:13     ` John Hubbard
2019-11-25  4:20 ` [PATCH 08/19] goldish_pipe: convert to pin_user_pages() and put_user_page() John Hubbard
2019-11-25  4:20 ` [PATCH 09/19] IB/{core,hw,umem}: set FOLL_PIN via pin_user_pages*(), fix up ODP John Hubbard
2019-11-25  4:20 ` [PATCH 10/19] mm/process_vm_access: set FOLL_PIN via pin_user_pages_remote() John Hubbard
2019-11-25  4:20 ` [PATCH 11/19] drm/via: set FOLL_PIN via pin_user_pages_fast() John Hubbard
2019-11-25  4:20 ` [PATCH 12/19] fs/io_uring: set FOLL_PIN via pin_user_pages() John Hubbard
2019-11-25  4:20 ` [PATCH 13/19] net/xdp: " John Hubbard
2019-11-25  4:20 ` [PATCH 14/19] media/v4l2-core: set pages dirty upon releasing DMA buffers John Hubbard
2019-11-25  4:20 ` [PATCH 15/19] media/v4l2-core: pin_user_pages (FOLL_PIN) and put_user_page() conversion John Hubbard
2019-11-25  4:20 ` [PATCH 16/19] vfio, mm: " John Hubbard
2019-11-25  4:20 ` [PATCH 17/19] powerpc: book3s64: convert to pin_user_pages() and put_user_page() John Hubbard
2019-11-25  8:59   ` Jan Kara
2019-11-25 20:46     ` John Hubbard
2019-11-25  4:20 ` [PATCH 18/19] mm/gup_benchmark: use proper FOLL_WRITE flags instead of hard-coding "1" John Hubbard
2019-11-25  4:20 ` [PATCH 19/19] mm, tree-wide: rename put_user_page*() to unpin_user_page*() John Hubbard

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git