* [PATCH v8 00/14] MAP_DIRECT for DAX RDMA and userspace flush
@ 2017-10-10 14:48 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:48 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Dave Chinner, J. Bruce Fields, linux-mm, Sean Hefty,
Jeff Layton, Marek Szyprowski, Ashok Raj, Darrick J. Wong,
linux-rdma, Joerg Roedel, Doug Ledford, Christoph Hellwig,
Linus Torvalds, Jeff Moyer, Ross Zwisler, Hal Rosenstock,
Arnd Bergmann, Robin Murphy, Alexander Viro, Andy Lutomirski,
Greg Kroah-Hartman, linux-xfs, iommu, linux-api, linux-fsdevel,
Andrew Morton, David Woodhouse
Changes since v7 [1]:
* Fix IOVA reuse race by leaving the dma scatterlist mapped until
unregistration time. Use iommu_unmap() in ib_umem_lease_break() to
force-invalidate the ibverbs memory registration. (David Woodhouse)
* Introduce iomap_can_allocate() as a way to check if any layouts are
present in the mmap write-fault path to prevent block map changes, and
start the leak break process when an allocating write-fault occurs.
This also removes the i_mapdcount bloat of 'struct inode' from v7.
(Dave Chinner)
* Provide generic_map_direct_{open,close,lease} to cleanup the
filesystem wiring to implement MAP_DIRECT support (Dave Chinner)
* Abandon (defer to a potential new fcntl()) support for using
MAP_DIRECT on non-DAX files. With this change we can validate the
inode is MAP_DIRECT capable just once at mmap time rather than every
fault. (Dave Chinner)
* Arrange for lease_direct leases to also wait the
/proc/sys/fs/lease-break-time period before calling break_fn. For
example, allow the lease-holder time to quiesce RDMA operations before
the iommu starts throwing io-faults.
* Switch intel-iommu to use iommu_num_sg_pages().
[1]: https://lists.01.org/pipermail/linux-nvdimm/2017-October/012707.html
---
MAP_DIRECT is a mechanism that allows an application to establish a
mapping where the kernel will not change the block-map, or otherwise
dirty the block-map metadata of a file without notification. It supports
a "flush from userspace" model where persistent memory applications can
bypass the overhead of ongoing coordination of writes with the
filesystem, and it provides safety to RDMA operations involving DAX
mappings.
The kernel always has the ability to revoke access and convert the file
back to normal operation after performing a "lease break". Similar to
fcntl leases, there is no way for userspace to to cancel the lease break
process once it has started, it can only delay it via the
/proc/sys/fs/lease-break-time setting.
MAP_DIRECT enables XFS to supplant the device-dax interface for
mmap-write access to persistent memory with no ongoing coordination with
the filesystem via fsync/msync syscalls.
---
Dan Williams (14):
mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
fs, mm: pass fd to ->mmap_validate()
fs: MAP_DIRECT core
xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
fs, xfs, iomap: introduce iomap_can_allocate()
xfs: wire up MAP_DIRECT
iommu, dma-mapping: introduce dma_get_iommu_domain()
fs, mapdirect: introduce ->lease_direct()
xfs: wire up ->lease_direct()
device-dax: wire up ->lease_direct()
iommu: up-level sg_num_pages() from amd-iommu
iommu/vt-d: use iommu_num_sg_pages
IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
tools/testing/nvdimm: enable rdma unit tests
arch/alpha/include/uapi/asm/mman.h | 1
arch/mips/include/uapi/asm/mman.h | 1
arch/mips/kernel/vdso.c | 2
arch/parisc/include/uapi/asm/mman.h | 1
arch/tile/mm/elf.c | 3
arch/x86/mm/mpx.c | 3
arch/xtensa/include/uapi/asm/mman.h | 1
drivers/base/dma-mapping.c | 10 +
drivers/dax/Kconfig | 1
drivers/dax/device.c | 4
drivers/infiniband/core/umem.c | 90 +++++-
drivers/iommu/amd_iommu.c | 40 +--
drivers/iommu/intel-iommu.c | 30 +-
drivers/iommu/iommu.c | 27 ++
fs/Kconfig | 5
fs/Makefile | 1
fs/aio.c | 2
fs/mapdirect.c | 382 ++++++++++++++++++++++++++
fs/xfs/Kconfig | 4
fs/xfs/Makefile | 1
fs/xfs/xfs_file.c | 103 +++++++
fs/xfs/xfs_iomap.c | 3
fs/xfs/xfs_layout.c | 45 +++
fs/xfs/xfs_layout.h | 13 +
fs/xfs/xfs_pnfs.c | 30 --
fs/xfs/xfs_pnfs.h | 10 -
include/linux/dma-mapping.h | 3
include/linux/fs.h | 2
include/linux/iomap.h | 10 +
include/linux/iommu.h | 2
include/linux/mapdirect.h | 57 ++++
include/linux/mm.h | 17 +
include/linux/mman.h | 42 +++
include/rdma/ib_umem.h | 8 +
include/uapi/asm-generic/mman-common.h | 1
include/uapi/asm-generic/mman.h | 1
ipc/shm.c | 3
mm/internal.h | 2
mm/mmap.c | 28 +-
mm/nommu.c | 5
mm/util.c | 7
tools/include/uapi/asm-generic/mman-common.h | 1
tools/testing/nvdimm/Kbuild | 31 ++
tools/testing/nvdimm/config_check.c | 2
tools/testing/nvdimm/test/iomap.c | 14 +
45 files changed, 938 insertions(+), 111 deletions(-)
create mode 100644 fs/mapdirect.c
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
create mode 100644 include/linux/mapdirect.h
^ permalink raw reply [flat|nested] 77+ messages in thread* [PATCH v8 00/14] MAP_DIRECT for DAX RDMA and userspace flush
@ 2017-10-10 14:48 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:48 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Dave Chinner, J. Bruce Fields, linux-mm, Sean Hefty,
Jeff Layton, Marek Szyprowski, Ashok Raj, Darrick J. Wong,
linux-rdma, Joerg Roedel, Doug Ledford, Christoph Hellwig,
Linus Torvalds, Jeff Moyer, Ross Zwisler, Hal Rosenstock,
Arnd Bergmann, Robin Murphy, Alexander Viro, Andy Lutomirski,
Greg Kroah-Hartman, linux-xfs, iommu, linux-api, linux-fsdevel,
Andrew Morton, David Woodhouse
Changes since v7 [1]:
* Fix IOVA reuse race by leaving the dma scatterlist mapped until
unregistration time. Use iommu_unmap() in ib_umem_lease_break() to
force-invalidate the ibverbs memory registration. (David Woodhouse)
* Introduce iomap_can_allocate() as a way to check if any layouts are
present in the mmap write-fault path to prevent block map changes, and
start the leak break process when an allocating write-fault occurs.
This also removes the i_mapdcount bloat of 'struct inode' from v7.
(Dave Chinner)
* Provide generic_map_direct_{open,close,lease} to cleanup the
filesystem wiring to implement MAP_DIRECT support (Dave Chinner)
* Abandon (defer to a potential new fcntl()) support for using
MAP_DIRECT on non-DAX files. With this change we can validate the
inode is MAP_DIRECT capable just once at mmap time rather than every
fault. (Dave Chinner)
* Arrange for lease_direct leases to also wait the
/proc/sys/fs/lease-break-time period before calling break_fn. For
example, allow the lease-holder time to quiesce RDMA operations before
the iommu starts throwing io-faults.
* Switch intel-iommu to use iommu_num_sg_pages().
[1]: https://lists.01.org/pipermail/linux-nvdimm/2017-October/012707.html
---
MAP_DIRECT is a mechanism that allows an application to establish a
mapping where the kernel will not change the block-map, or otherwise
dirty the block-map metadata of a file without notification. It supports
a "flush from userspace" model where persistent memory applications can
bypass the overhead of ongoing coordination of writes with the
filesystem, and it provides safety to RDMA operations involving DAX
mappings.
The kernel always has the ability to revoke access and convert the file
back to normal operation after performing a "lease break". Similar to
fcntl leases, there is no way for userspace to to cancel the lease break
process once it has started, it can only delay it via the
/proc/sys/fs/lease-break-time setting.
MAP_DIRECT enables XFS to supplant the device-dax interface for
mmap-write access to persistent memory with no ongoing coordination with
the filesystem via fsync/msync syscalls.
---
Dan Williams (14):
mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
fs, mm: pass fd to ->mmap_validate()
fs: MAP_DIRECT core
xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
fs, xfs, iomap: introduce iomap_can_allocate()
xfs: wire up MAP_DIRECT
iommu, dma-mapping: introduce dma_get_iommu_domain()
fs, mapdirect: introduce ->lease_direct()
xfs: wire up ->lease_direct()
device-dax: wire up ->lease_direct()
iommu: up-level sg_num_pages() from amd-iommu
iommu/vt-d: use iommu_num_sg_pages
IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
tools/testing/nvdimm: enable rdma unit tests
arch/alpha/include/uapi/asm/mman.h | 1
arch/mips/include/uapi/asm/mman.h | 1
arch/mips/kernel/vdso.c | 2
arch/parisc/include/uapi/asm/mman.h | 1
arch/tile/mm/elf.c | 3
arch/x86/mm/mpx.c | 3
arch/xtensa/include/uapi/asm/mman.h | 1
drivers/base/dma-mapping.c | 10 +
drivers/dax/Kconfig | 1
drivers/dax/device.c | 4
drivers/infiniband/core/umem.c | 90 +++++-
drivers/iommu/amd_iommu.c | 40 +--
drivers/iommu/intel-iommu.c | 30 +-
drivers/iommu/iommu.c | 27 ++
fs/Kconfig | 5
fs/Makefile | 1
fs/aio.c | 2
fs/mapdirect.c | 382 ++++++++++++++++++++++++++
fs/xfs/Kconfig | 4
fs/xfs/Makefile | 1
fs/xfs/xfs_file.c | 103 +++++++
fs/xfs/xfs_iomap.c | 3
fs/xfs/xfs_layout.c | 45 +++
fs/xfs/xfs_layout.h | 13 +
fs/xfs/xfs_pnfs.c | 30 --
fs/xfs/xfs_pnfs.h | 10 -
include/linux/dma-mapping.h | 3
include/linux/fs.h | 2
include/linux/iomap.h | 10 +
include/linux/iommu.h | 2
include/linux/mapdirect.h | 57 ++++
include/linux/mm.h | 17 +
include/linux/mman.h | 42 +++
include/rdma/ib_umem.h | 8 +
include/uapi/asm-generic/mman-common.h | 1
include/uapi/asm-generic/mman.h | 1
ipc/shm.c | 3
mm/internal.h | 2
mm/mmap.c | 28 +-
mm/nommu.c | 5
mm/util.c | 7
tools/include/uapi/asm-generic/mman-common.h | 1
tools/testing/nvdimm/Kbuild | 31 ++
tools/testing/nvdimm/config_check.c | 2
tools/testing/nvdimm/test/iomap.c | 14 +
45 files changed, 938 insertions(+), 111 deletions(-)
create mode 100644 fs/mapdirect.c
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
create mode 100644 include/linux/mapdirect.h
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread* [PATCH v8 00/14] MAP_DIRECT for DAX RDMA and userspace flush
@ 2017-10-10 14:48 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:48 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: Jan Kara, Dave Chinner, J. Bruce Fields,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Sean Hefty, Christoph Hellwig,
Marek Szyprowski, Ashok Raj, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Joerg Roedel, Doug Ledford,
Linus Torvalds, Hal Rosenstock, Arnd Bergmann, Alexander Viro,
Andy Lutomirski, Jeff Layton, Greg Kroah-Hartman,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Andrew Morton,
David Woodhouse
Changes since v7 [1]:
* Fix IOVA reuse race by leaving the dma scatterlist mapped until
unregistration time. Use iommu_unmap() in ib_umem_lease_break() to
force-invalidate the ibverbs memory registration. (David Woodhouse)
* Introduce iomap_can_allocate() as a way to check if any layouts are
present in the mmap write-fault path to prevent block map changes, and
start the leak break process when an allocating write-fault occurs.
This also removes the i_mapdcount bloat of 'struct inode' from v7.
(Dave Chinner)
* Provide generic_map_direct_{open,close,lease} to cleanup the
filesystem wiring to implement MAP_DIRECT support (Dave Chinner)
* Abandon (defer to a potential new fcntl()) support for using
MAP_DIRECT on non-DAX files. With this change we can validate the
inode is MAP_DIRECT capable just once at mmap time rather than every
fault. (Dave Chinner)
* Arrange for lease_direct leases to also wait the
/proc/sys/fs/lease-break-time period before calling break_fn. For
example, allow the lease-holder time to quiesce RDMA operations before
the iommu starts throwing io-faults.
* Switch intel-iommu to use iommu_num_sg_pages().
[1]: https://lists.01.org/pipermail/linux-nvdimm/2017-October/012707.html
---
MAP_DIRECT is a mechanism that allows an application to establish a
mapping where the kernel will not change the block-map, or otherwise
dirty the block-map metadata of a file without notification. It supports
a "flush from userspace" model where persistent memory applications can
bypass the overhead of ongoing coordination of writes with the
filesystem, and it provides safety to RDMA operations involving DAX
mappings.
The kernel always has the ability to revoke access and convert the file
back to normal operation after performing a "lease break". Similar to
fcntl leases, there is no way for userspace to to cancel the lease break
process once it has started, it can only delay it via the
/proc/sys/fs/lease-break-time setting.
MAP_DIRECT enables XFS to supplant the device-dax interface for
mmap-write access to persistent memory with no ongoing coordination with
the filesystem via fsync/msync syscalls.
---
Dan Williams (14):
mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
fs, mm: pass fd to ->mmap_validate()
fs: MAP_DIRECT core
xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
fs, xfs, iomap: introduce iomap_can_allocate()
xfs: wire up MAP_DIRECT
iommu, dma-mapping: introduce dma_get_iommu_domain()
fs, mapdirect: introduce ->lease_direct()
xfs: wire up ->lease_direct()
device-dax: wire up ->lease_direct()
iommu: up-level sg_num_pages() from amd-iommu
iommu/vt-d: use iommu_num_sg_pages
IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
tools/testing/nvdimm: enable rdma unit tests
arch/alpha/include/uapi/asm/mman.h | 1
arch/mips/include/uapi/asm/mman.h | 1
arch/mips/kernel/vdso.c | 2
arch/parisc/include/uapi/asm/mman.h | 1
arch/tile/mm/elf.c | 3
arch/x86/mm/mpx.c | 3
arch/xtensa/include/uapi/asm/mman.h | 1
drivers/base/dma-mapping.c | 10 +
drivers/dax/Kconfig | 1
drivers/dax/device.c | 4
drivers/infiniband/core/umem.c | 90 +++++-
drivers/iommu/amd_iommu.c | 40 +--
drivers/iommu/intel-iommu.c | 30 +-
drivers/iommu/iommu.c | 27 ++
fs/Kconfig | 5
fs/Makefile | 1
fs/aio.c | 2
fs/mapdirect.c | 382 ++++++++++++++++++++++++++
fs/xfs/Kconfig | 4
fs/xfs/Makefile | 1
fs/xfs/xfs_file.c | 103 +++++++
fs/xfs/xfs_iomap.c | 3
fs/xfs/xfs_layout.c | 45 +++
fs/xfs/xfs_layout.h | 13 +
fs/xfs/xfs_pnfs.c | 30 --
fs/xfs/xfs_pnfs.h | 10 -
include/linux/dma-mapping.h | 3
include/linux/fs.h | 2
include/linux/iomap.h | 10 +
include/linux/iommu.h | 2
include/linux/mapdirect.h | 57 ++++
include/linux/mm.h | 17 +
include/linux/mman.h | 42 +++
include/rdma/ib_umem.h | 8 +
include/uapi/asm-generic/mman-common.h | 1
include/uapi/asm-generic/mman.h | 1
ipc/shm.c | 3
mm/internal.h | 2
mm/mmap.c | 28 +-
mm/nommu.c | 5
mm/util.c | 7
tools/include/uapi/asm-generic/mman-common.h | 1
tools/testing/nvdimm/Kbuild | 31 ++
tools/testing/nvdimm/config_check.c | 2
tools/testing/nvdimm/test/iomap.c | 14 +
45 files changed, 938 insertions(+), 111 deletions(-)
create mode 100644 fs/mapdirect.c
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
create mode 100644 include/linux/mapdirect.h
^ permalink raw reply [flat|nested] 77+ messages in thread* [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Arnd Bergmann, linux-rdma, linux-api, linux-xfs,
linux-mm, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
The mmap(2) syscall suffers from the ABI anti-pattern of not validating
unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
mechanism to define new behavior that is known to fail on older kernels
without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
is guaranteed to fail on all legacy mmap implementations.
It is worth noting that the original proposal was for a standalone
MAP_VALIDATE flag. However, when that could not be supported by all
archs Linus observed:
I see why you *think* you want a bitmap. You think you want
a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
etc, so that people can do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
| MAP_SYNC, fd, 0);
and "know" that MAP_SYNC actually takes.
And I'm saying that whole wish is bogus. You're fundamentally
depending on special semantics, just make it explicit. It's already
not portable, so don't try to make it so.
Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
of 0x3, and make people do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
| MAP_SYNC, fd, 0);
and then the kernel side is easier too (none of that random garbage
playing games with looking at the "MAP_VALIDATE bit", but just another
case statement in that map type thing.
Boom. Done.
Similar to ->fallocate() we also want the ability to validate the
support for new flags on a per ->mmap() 'struct file_operations'
instance basis. Towards that end arrange for flags to be generically
validated against a mmap_supported_mask exported by 'struct
file_operations'. By default all existing flags are implicitly
supported, but new flags require MAP_SHARED_VALIDATE and
per-instance-opt-in.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/mips/kernel/vdso.c | 2 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/tile/mm/elf.c | 3 +-
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/linux/fs.h | 2 +
include/linux/mm.h | 2 +
include/linux/mman.h | 39 ++++++++++++++++++++++++++
include/uapi/asm-generic/mman-common.h | 1 +
mm/mmap.c | 21 ++++++++++++--
tools/include/uapi/asm-generic/mman-common.h | 1 +
12 files changed, 69 insertions(+), 6 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 3b26cc62dadb..92823f24890b 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
#define MAP_FIXED 0x100 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with OSF/1 defines */
#define _MAP_HASSEMAPHORE 0x0200
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index da3216007fe0..c77689076577 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -30,6 +30,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 019035d7225c..cf10654477a9 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL);
+ 0, NULL, 0);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 775b5d5e41a1..36b688d52de3 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x03 /* Mask for type of mapping */
#define MAP_FIXED 0x04 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 889901824400..5ffcbe76aef9 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -143,7 +143,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
unsigned long addr = MEM_USER_INTRPT;
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0, NULL);
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
+ NULL, 0);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index b15b278aa314..ec597900eec7 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -37,6 +37,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 339e73742e73..51538958f7f5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1701,6 +1701,8 @@ struct file_operations {
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
+ int (*mmap_validate) (struct file *, struct vm_area_struct *,
+ unsigned long);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index f8c10d336e42..5c4c98e4adc9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf);
+ struct list_head *uf, unsigned long map_flags);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index c8367041fafd..94b63b4d71ff 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -7,6 +7,45 @@
#include <linux/atomic.h>
#include <uapi/linux/mman.h>
+/*
+ * Arrange for legacy / undefined architecture specific flags to be
+ * ignored by default in LEGACY_MAP_MASK.
+ */
+#ifndef MAP_32BIT
+#define MAP_32BIT 0
+#endif
+#ifndef MAP_HUGE_2MB
+#define MAP_HUGE_2MB 0
+#endif
+#ifndef MAP_HUGE_1GB
+#define MAP_HUGE_1GB 0
+#endif
+#ifndef MAP_UNINITIALIZED
+#define MAP_UNINITIALIZED 0
+#endif
+
+/*
+ * The historical set of flags that all mmap implementations implicitly
+ * support when a ->mmap_validate() op is not provided in file_operations.
+ */
+#define LEGACY_MAP_MASK (MAP_SHARED \
+ | MAP_PRIVATE \
+ | MAP_FIXED \
+ | MAP_ANONYMOUS \
+ | MAP_DENYWRITE \
+ | MAP_EXECUTABLE \
+ | MAP_UNINITIALIZED \
+ | MAP_GROWSDOWN \
+ | MAP_LOCKED \
+ | MAP_NORESERVE \
+ | MAP_POPULATE \
+ | MAP_NONBLOCK \
+ | MAP_STACK \
+ | MAP_HUGETLB \
+ | MAP_32BIT \
+ | MAP_HUGE_2MB \
+ | MAP_HUGE_1GB)
+
extern int sysctl_overcommit_memory;
extern int sysctl_overcommit_ratio;
extern unsigned long sysctl_overcommit_kbytes;
diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
index 203268f9231e..ac55d1c0ec0f 100644
--- a/include/uapi/asm-generic/mman-common.h
+++ b/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
diff --git a/mm/mmap.c b/mm/mmap.c
index 680506faceae..a1bcaa9eff42 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1389,6 +1389,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
struct inode *inode = file_inode(file);
switch (flags & MAP_TYPE) {
+ case (MAP_SHARED_VALIDATE):
+ if ((flags & ~LEGACY_MAP_MASK) == 0) {
+ /*
+ * If all legacy mmap flags, downgrade
+ * to MAP_SHARED, i.e. invoke ->mmap()
+ * instead of ->mmap_validate()
+ */
+ flags &= ~MAP_TYPE;
+ flags |= MAP_SHARED;
+ } else if (!file->f_op->mmap_validate)
+ return -EOPNOTSUPP;
+ /* fall through */
case MAP_SHARED:
if ((prot&PROT_WRITE) && !(file->f_mode&FMODE_WRITE))
return -EACCES;
@@ -1465,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1602,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf)
+ struct list_head *uf, unsigned long map_flags)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1687,7 +1699,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
* new file must not have been exposed to user-space, yet.
*/
vma->vm_file = get_file(file);
- error = call_mmap(file, vma);
+ if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
+ error = file->f_op->mmap_validate(file, vma, map_flags);
+ else
+ error = call_mmap(file, vma);
if (error)
goto unmap_and_free_vma;
diff --git a/tools/include/uapi/asm-generic/mman-common.h b/tools/include/uapi/asm-generic/mman-common.h
index 8c27db0c5c08..202bc4277fb5 100644
--- a/tools/include/uapi/asm-generic/mman-common.h
+++ b/tools/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Arnd Bergmann, linux-rdma, linux-api, linux-xfs,
linux-mm, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
The mmap(2) syscall suffers from the ABI anti-pattern of not validating
unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
mechanism to define new behavior that is known to fail on older kernels
without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
is guaranteed to fail on all legacy mmap implementations.
It is worth noting that the original proposal was for a standalone
MAP_VALIDATE flag. However, when that could not be supported by all
archs Linus observed:
I see why you *think* you want a bitmap. You think you want
a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
etc, so that people can do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
| MAP_SYNC, fd, 0);
and "know" that MAP_SYNC actually takes.
And I'm saying that whole wish is bogus. You're fundamentally
depending on special semantics, just make it explicit. It's already
not portable, so don't try to make it so.
Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
of 0x3, and make people do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
| MAP_SYNC, fd, 0);
and then the kernel side is easier too (none of that random garbage
playing games with looking at the "MAP_VALIDATE bit", but just another
case statement in that map type thing.
Boom. Done.
Similar to ->fallocate() we also want the ability to validate the
support for new flags on a per ->mmap() 'struct file_operations'
instance basis. Towards that end arrange for flags to be generically
validated against a mmap_supported_mask exported by 'struct
file_operations'. By default all existing flags are implicitly
supported, but new flags require MAP_SHARED_VALIDATE and
per-instance-opt-in.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/mips/kernel/vdso.c | 2 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/tile/mm/elf.c | 3 +-
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/linux/fs.h | 2 +
include/linux/mm.h | 2 +
include/linux/mman.h | 39 ++++++++++++++++++++++++++
include/uapi/asm-generic/mman-common.h | 1 +
mm/mmap.c | 21 ++++++++++++--
tools/include/uapi/asm-generic/mman-common.h | 1 +
12 files changed, 69 insertions(+), 6 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 3b26cc62dadb..92823f24890b 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
#define MAP_FIXED 0x100 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with OSF/1 defines */
#define _MAP_HASSEMAPHORE 0x0200
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index da3216007fe0..c77689076577 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -30,6 +30,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 019035d7225c..cf10654477a9 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL);
+ 0, NULL, 0);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 775b5d5e41a1..36b688d52de3 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x03 /* Mask for type of mapping */
#define MAP_FIXED 0x04 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 889901824400..5ffcbe76aef9 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -143,7 +143,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
unsigned long addr = MEM_USER_INTRPT;
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0, NULL);
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
+ NULL, 0);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index b15b278aa314..ec597900eec7 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -37,6 +37,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 339e73742e73..51538958f7f5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1701,6 +1701,8 @@ struct file_operations {
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
+ int (*mmap_validate) (struct file *, struct vm_area_struct *,
+ unsigned long);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index f8c10d336e42..5c4c98e4adc9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf);
+ struct list_head *uf, unsigned long map_flags);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index c8367041fafd..94b63b4d71ff 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -7,6 +7,45 @@
#include <linux/atomic.h>
#include <uapi/linux/mman.h>
+/*
+ * Arrange for legacy / undefined architecture specific flags to be
+ * ignored by default in LEGACY_MAP_MASK.
+ */
+#ifndef MAP_32BIT
+#define MAP_32BIT 0
+#endif
+#ifndef MAP_HUGE_2MB
+#define MAP_HUGE_2MB 0
+#endif
+#ifndef MAP_HUGE_1GB
+#define MAP_HUGE_1GB 0
+#endif
+#ifndef MAP_UNINITIALIZED
+#define MAP_UNINITIALIZED 0
+#endif
+
+/*
+ * The historical set of flags that all mmap implementations implicitly
+ * support when a ->mmap_validate() op is not provided in file_operations.
+ */
+#define LEGACY_MAP_MASK (MAP_SHARED \
+ | MAP_PRIVATE \
+ | MAP_FIXED \
+ | MAP_ANONYMOUS \
+ | MAP_DENYWRITE \
+ | MAP_EXECUTABLE \
+ | MAP_UNINITIALIZED \
+ | MAP_GROWSDOWN \
+ | MAP_LOCKED \
+ | MAP_NORESERVE \
+ | MAP_POPULATE \
+ | MAP_NONBLOCK \
+ | MAP_STACK \
+ | MAP_HUGETLB \
+ | MAP_32BIT \
+ | MAP_HUGE_2MB \
+ | MAP_HUGE_1GB)
+
extern int sysctl_overcommit_memory;
extern int sysctl_overcommit_ratio;
extern unsigned long sysctl_overcommit_kbytes;
diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
index 203268f9231e..ac55d1c0ec0f 100644
--- a/include/uapi/asm-generic/mman-common.h
+++ b/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
diff --git a/mm/mmap.c b/mm/mmap.c
index 680506faceae..a1bcaa9eff42 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1389,6 +1389,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
struct inode *inode = file_inode(file);
switch (flags & MAP_TYPE) {
+ case (MAP_SHARED_VALIDATE):
+ if ((flags & ~LEGACY_MAP_MASK) == 0) {
+ /*
+ * If all legacy mmap flags, downgrade
+ * to MAP_SHARED, i.e. invoke ->mmap()
+ * instead of ->mmap_validate()
+ */
+ flags &= ~MAP_TYPE;
+ flags |= MAP_SHARED;
+ } else if (!file->f_op->mmap_validate)
+ return -EOPNOTSUPP;
+ /* fall through */
case MAP_SHARED:
if ((prot&PROT_WRITE) && !(file->f_mode&FMODE_WRITE))
return -EACCES;
@@ -1465,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1602,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf)
+ struct list_head *uf, unsigned long map_flags)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1687,7 +1699,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
* new file must not have been exposed to user-space, yet.
*/
vma->vm_file = get_file(file);
- error = call_mmap(file, vma);
+ if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
+ error = file->f_op->mmap_validate(file, vma, map_flags);
+ else
+ error = call_mmap(file, vma);
if (error)
goto unmap_and_free_vma;
diff --git a/tools/include/uapi/asm-generic/mman-common.h b/tools/include/uapi/asm-generic/mman-common.h
index 8c27db0c5c08..202bc4277fb5 100644
--- a/tools/include/uapi/asm-generic/mman-common.h
+++ b/tools/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Arnd Bergmann, linux-rdma, linux-api, linux-xfs,
linux-mm, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
The mmap(2) syscall suffers from the ABI anti-pattern of not validating
unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
mechanism to define new behavior that is known to fail on older kernels
without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
is guaranteed to fail on all legacy mmap implementations.
It is worth noting that the original proposal was for a standalone
MAP_VALIDATE flag. However, when that could not be supported by all
archs Linus observed:
I see why you *think* you want a bitmap. You think you want
a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
etc, so that people can do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
| MAP_SYNC, fd, 0);
and "know" that MAP_SYNC actually takes.
And I'm saying that whole wish is bogus. You're fundamentally
depending on special semantics, just make it explicit. It's already
not portable, so don't try to make it so.
Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
of 0x3, and make people do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
| MAP_SYNC, fd, 0);
and then the kernel side is easier too (none of that random garbage
playing games with looking at the "MAP_VALIDATE bit", but just another
case statement in that map type thing.
Boom. Done.
Similar to ->fallocate() we also want the ability to validate the
support for new flags on a per ->mmap() 'struct file_operations'
instance basis. Towards that end arrange for flags to be generically
validated against a mmap_supported_mask exported by 'struct
file_operations'. By default all existing flags are implicitly
supported, but new flags require MAP_SHARED_VALIDATE and
per-instance-opt-in.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/mips/kernel/vdso.c | 2 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/tile/mm/elf.c | 3 +-
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/linux/fs.h | 2 +
include/linux/mm.h | 2 +
include/linux/mman.h | 39 ++++++++++++++++++++++++++
include/uapi/asm-generic/mman-common.h | 1 +
mm/mmap.c | 21 ++++++++++++--
tools/include/uapi/asm-generic/mman-common.h | 1 +
12 files changed, 69 insertions(+), 6 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 3b26cc62dadb..92823f24890b 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
#define MAP_FIXED 0x100 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with OSF/1 defines */
#define _MAP_HASSEMAPHORE 0x0200
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index da3216007fe0..c77689076577 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -30,6 +30,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 019035d7225c..cf10654477a9 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL);
+ 0, NULL, 0);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 775b5d5e41a1..36b688d52de3 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x03 /* Mask for type of mapping */
#define MAP_FIXED 0x04 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 889901824400..5ffcbe76aef9 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -143,7 +143,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
unsigned long addr = MEM_USER_INTRPT;
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0, NULL);
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
+ NULL, 0);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index b15b278aa314..ec597900eec7 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -37,6 +37,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 339e73742e73..51538958f7f5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1701,6 +1701,8 @@ struct file_operations {
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
+ int (*mmap_validate) (struct file *, struct vm_area_struct *,
+ unsigned long);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index f8c10d336e42..5c4c98e4adc9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf);
+ struct list_head *uf, unsigned long map_flags);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index c8367041fafd..94b63b4d71ff 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -7,6 +7,45 @@
#include <linux/atomic.h>
#include <uapi/linux/mman.h>
+/*
+ * Arrange for legacy / undefined architecture specific flags to be
+ * ignored by default in LEGACY_MAP_MASK.
+ */
+#ifndef MAP_32BIT
+#define MAP_32BIT 0
+#endif
+#ifndef MAP_HUGE_2MB
+#define MAP_HUGE_2MB 0
+#endif
+#ifndef MAP_HUGE_1GB
+#define MAP_HUGE_1GB 0
+#endif
+#ifndef MAP_UNINITIALIZED
+#define MAP_UNINITIALIZED 0
+#endif
+
+/*
+ * The historical set of flags that all mmap implementations implicitly
+ * support when a ->mmap_validate() op is not provided in file_operations.
+ */
+#define LEGACY_MAP_MASK (MAP_SHARED \
+ | MAP_PRIVATE \
+ | MAP_FIXED \
+ | MAP_ANONYMOUS \
+ | MAP_DENYWRITE \
+ | MAP_EXECUTABLE \
+ | MAP_UNINITIALIZED \
+ | MAP_GROWSDOWN \
+ | MAP_LOCKED \
+ | MAP_NORESERVE \
+ | MAP_POPULATE \
+ | MAP_NONBLOCK \
+ | MAP_STACK \
+ | MAP_HUGETLB \
+ | MAP_32BIT \
+ | MAP_HUGE_2MB \
+ | MAP_HUGE_1GB)
+
extern int sysctl_overcommit_memory;
extern int sysctl_overcommit_ratio;
extern unsigned long sysctl_overcommit_kbytes;
diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
index 203268f9231e..ac55d1c0ec0f 100644
--- a/include/uapi/asm-generic/mman-common.h
+++ b/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
diff --git a/mm/mmap.c b/mm/mmap.c
index 680506faceae..a1bcaa9eff42 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1389,6 +1389,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
struct inode *inode = file_inode(file);
switch (flags & MAP_TYPE) {
+ case (MAP_SHARED_VALIDATE):
+ if ((flags & ~LEGACY_MAP_MASK) == 0) {
+ /*
+ * If all legacy mmap flags, downgrade
+ * to MAP_SHARED, i.e. invoke ->mmap()
+ * instead of ->mmap_validate()
+ */
+ flags &= ~MAP_TYPE;
+ flags |= MAP_SHARED;
+ } else if (!file->f_op->mmap_validate)
+ return -EOPNOTSUPP;
+ /* fall through */
case MAP_SHARED:
if ((prot&PROT_WRITE) && !(file->f_mode&FMODE_WRITE))
return -EACCES;
@@ -1465,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1602,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf)
+ struct list_head *uf, unsigned long map_flags)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1687,7 +1699,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
* new file must not have been exposed to user-space, yet.
*/
vma->vm_file = get_file(file);
- error = call_mmap(file, vma);
+ if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
+ error = file->f_op->mmap_validate(file, vma, map_flags);
+ else
+ error = call_mmap(file, vma);
if (error)
goto unmap_and_free_vma;
diff --git a/tools/include/uapi/asm-generic/mman-common.h b/tools/include/uapi/asm-generic/mman-common.h
index 8c27db0c5c08..202bc4277fb5 100644
--- a/tools/include/uapi/asm-generic/mman-common.h
+++ b/tools/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: Jan Kara, Arnd Bergmann, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Andy Lutomirski, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
Andrew Morton, Linus Torvalds, Christoph Hellwig
The mmap(2) syscall suffers from the ABI anti-pattern of not validating
unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
mechanism to define new behavior that is known to fail on older kernels
without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
is guaranteed to fail on all legacy mmap implementations.
It is worth noting that the original proposal was for a standalone
MAP_VALIDATE flag. However, when that could not be supported by all
archs Linus observed:
I see why you *think* you want a bitmap. You think you want
a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
etc, so that people can do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
| MAP_SYNC, fd, 0);
and "know" that MAP_SYNC actually takes.
And I'm saying that whole wish is bogus. You're fundamentally
depending on special semantics, just make it explicit. It's already
not portable, so don't try to make it so.
Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
of 0x3, and make people do
ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
| MAP_SYNC, fd, 0);
and then the kernel side is easier too (none of that random garbage
playing games with looking at the "MAP_VALIDATE bit", but just another
case statement in that map type thing.
Boom. Done.
Similar to ->fallocate() we also want the ability to validate the
support for new flags on a per ->mmap() 'struct file_operations'
instance basis. Towards that end arrange for flags to be generically
validated against a mmap_supported_mask exported by 'struct
file_operations'. By default all existing flags are implicitly
supported, but new flags require MAP_SHARED_VALIDATE and
per-instance-opt-in.
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Cc: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
Cc: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
Suggested-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Suggested-by: Linus Torvalds <torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/mips/kernel/vdso.c | 2 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/tile/mm/elf.c | 3 +-
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/linux/fs.h | 2 +
include/linux/mm.h | 2 +
include/linux/mman.h | 39 ++++++++++++++++++++++++++
include/uapi/asm-generic/mman-common.h | 1 +
mm/mmap.c | 21 ++++++++++++--
tools/include/uapi/asm-generic/mman-common.h | 1 +
12 files changed, 69 insertions(+), 6 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 3b26cc62dadb..92823f24890b 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
#define MAP_FIXED 0x100 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with OSF/1 defines */
#define _MAP_HASSEMAPHORE 0x0200
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index da3216007fe0..c77689076577 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -30,6 +30,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 019035d7225c..cf10654477a9 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL);
+ 0, NULL, 0);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 775b5d5e41a1..36b688d52de3 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -14,6 +14,7 @@
#define MAP_TYPE 0x03 /* Mask for type of mapping */
#define MAP_FIXED 0x04 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x10 /* don't use a file */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 889901824400..5ffcbe76aef9 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -143,7 +143,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
unsigned long addr = MEM_USER_INTRPT;
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0, NULL);
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
+ NULL, 0);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index b15b278aa314..ec597900eec7 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -37,6 +37,7 @@
#define MAP_PRIVATE 0x002 /* Changes are private */
#define MAP_TYPE 0x00f /* Mask for type of mapping */
#define MAP_FIXED 0x010 /* Interpret addr exactly */
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/* not used by linux, but here to make sure we don't clash with ABI defines */
#define MAP_RENAME 0x020 /* Assign page to file */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 339e73742e73..51538958f7f5 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1701,6 +1701,8 @@ struct file_operations {
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
+ int (*mmap_validate) (struct file *, struct vm_area_struct *,
+ unsigned long);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index f8c10d336e42..5c4c98e4adc9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf);
+ struct list_head *uf, unsigned long map_flags);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index c8367041fafd..94b63b4d71ff 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -7,6 +7,45 @@
#include <linux/atomic.h>
#include <uapi/linux/mman.h>
+/*
+ * Arrange for legacy / undefined architecture specific flags to be
+ * ignored by default in LEGACY_MAP_MASK.
+ */
+#ifndef MAP_32BIT
+#define MAP_32BIT 0
+#endif
+#ifndef MAP_HUGE_2MB
+#define MAP_HUGE_2MB 0
+#endif
+#ifndef MAP_HUGE_1GB
+#define MAP_HUGE_1GB 0
+#endif
+#ifndef MAP_UNINITIALIZED
+#define MAP_UNINITIALIZED 0
+#endif
+
+/*
+ * The historical set of flags that all mmap implementations implicitly
+ * support when a ->mmap_validate() op is not provided in file_operations.
+ */
+#define LEGACY_MAP_MASK (MAP_SHARED \
+ | MAP_PRIVATE \
+ | MAP_FIXED \
+ | MAP_ANONYMOUS \
+ | MAP_DENYWRITE \
+ | MAP_EXECUTABLE \
+ | MAP_UNINITIALIZED \
+ | MAP_GROWSDOWN \
+ | MAP_LOCKED \
+ | MAP_NORESERVE \
+ | MAP_POPULATE \
+ | MAP_NONBLOCK \
+ | MAP_STACK \
+ | MAP_HUGETLB \
+ | MAP_32BIT \
+ | MAP_HUGE_2MB \
+ | MAP_HUGE_1GB)
+
extern int sysctl_overcommit_memory;
extern int sysctl_overcommit_ratio;
extern unsigned long sysctl_overcommit_kbytes;
diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
index 203268f9231e..ac55d1c0ec0f 100644
--- a/include/uapi/asm-generic/mman-common.h
+++ b/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
diff --git a/mm/mmap.c b/mm/mmap.c
index 680506faceae..a1bcaa9eff42 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1389,6 +1389,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
struct inode *inode = file_inode(file);
switch (flags & MAP_TYPE) {
+ case (MAP_SHARED_VALIDATE):
+ if ((flags & ~LEGACY_MAP_MASK) == 0) {
+ /*
+ * If all legacy mmap flags, downgrade
+ * to MAP_SHARED, i.e. invoke ->mmap()
+ * instead of ->mmap_validate()
+ */
+ flags &= ~MAP_TYPE;
+ flags |= MAP_SHARED;
+ } else if (!file->f_op->mmap_validate)
+ return -EOPNOTSUPP;
+ /* fall through */
case MAP_SHARED:
if ((prot&PROT_WRITE) && !(file->f_mode&FMODE_WRITE))
return -EACCES;
@@ -1465,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1602,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf)
+ struct list_head *uf, unsigned long map_flags)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1687,7 +1699,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
* new file must not have been exposed to user-space, yet.
*/
vma->vm_file = get_file(file);
- error = call_mmap(file, vma);
+ if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
+ error = file->f_op->mmap_validate(file, vma, map_flags);
+ else
+ error = call_mmap(file, vma);
if (error)
goto unmap_and_free_vma;
diff --git a/tools/include/uapi/asm-generic/mman-common.h b/tools/include/uapi/asm-generic/mman-common.h
index 8c27db0c5c08..202bc4277fb5 100644
--- a/tools/include/uapi/asm-generic/mman-common.h
+++ b/tools/include/uapi/asm-generic/mman-common.h
@@ -24,6 +24,7 @@
#else
# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
#endif
+#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
/*
* Flags for mlock
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 7:43 ` Jan Kara
0 siblings, 0 replies; 77+ messages in thread
From: Jan Kara @ 2017-10-11 7:43 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Jan Kara, Arnd Bergmann, linux-rdma, linux-api,
linux-xfs, linux-mm, iommu, Andy Lutomirski, linux-fsdevel,
Andrew Morton, Linus Torvalds, Christoph Hellwig
On Tue 10-10-17 07:49:01, Dan Williams wrote:
> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
> mechanism to define new behavior that is known to fail on older kernels
> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
> is guaranteed to fail on all legacy mmap implementations.
>
> It is worth noting that the original proposal was for a standalone
> MAP_VALIDATE flag. However, when that could not be supported by all
> archs Linus observed:
>
> I see why you *think* you want a bitmap. You think you want
> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
> etc, so that people can do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
> | MAP_SYNC, fd, 0);
>
> and "know" that MAP_SYNC actually takes.
>
> And I'm saying that whole wish is bogus. You're fundamentally
> depending on special semantics, just make it explicit. It's already
> not portable, so don't try to make it so.
>
> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
> of 0x3, and make people do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
> | MAP_SYNC, fd, 0);
>
> and then the kernel side is easier too (none of that random garbage
> playing games with looking at the "MAP_VALIDATE bit", but just another
> case statement in that map type thing.
>
> Boom. Done.
>
> Similar to ->fallocate() we also want the ability to validate the
> support for new flags on a per ->mmap() 'struct file_operations'
> instance basis. Towards that end arrange for flags to be generically
> validated against a mmap_supported_mask exported by 'struct
> file_operations'. By default all existing flags are implicitly
> supported, but new flags require MAP_SHARED_VALIDATE and
> per-instance-opt-in.
>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/alpha/include/uapi/asm/mman.h | 1 +
> arch/mips/include/uapi/asm/mman.h | 1 +
> arch/mips/kernel/vdso.c | 2 +
> arch/parisc/include/uapi/asm/mman.h | 1 +
> arch/tile/mm/elf.c | 3 +-
> arch/xtensa/include/uapi/asm/mman.h | 1 +
> include/linux/fs.h | 2 +
> include/linux/mm.h | 2 +
> include/linux/mman.h | 39 ++++++++++++++++++++++++++
> include/uapi/asm-generic/mman-common.h | 1 +
> mm/mmap.c | 21 ++++++++++++--
> tools/include/uapi/asm-generic/mman-common.h | 1 +
> 12 files changed, 69 insertions(+), 6 deletions(-)
>
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..92823f24890b 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -14,6 +14,7 @@
> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
> #define MAP_FIXED 0x100 /* Interpret addr exactly */
> #define MAP_ANONYMOUS 0x10 /* don't use a file */
> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
all archs).
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index f8c10d336e42..5c4c98e4adc9 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>
> extern unsigned long mmap_region(struct file *file, unsigned long addr,
> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
> - struct list_head *uf);
> + struct list_head *uf, unsigned long map_flags);
> extern unsigned long do_mmap(struct file *file, unsigned long addr,
> unsigned long len, unsigned long prot, unsigned long flags,
> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
I have to say I'm not very keen on passing down both vm_flags and map_flags
- vm_flags are almost a subset of map_flags but not quite and the ambiguity
which needs to be used for a particular check seems to open a space for
errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
and just pass map_flags through mmap_region() so there's no space for
confusion but future checks could do something different. But OTOH I don't
see a cleaner way of avoiding the need to allocate vma flag for something
you need to check down in ->mmap_validate so I guess I'll live with that
and if problems really happen, we may have cleaner idea what needs to be
done.
So overall feel free to add:
Reviewed-by: Jan Kara <jack@suse.cz>
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 7:43 ` Jan Kara
0 siblings, 0 replies; 77+ messages in thread
From: Jan Kara @ 2017-10-11 7:43 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Jan Kara, Arnd Bergmann, linux-rdma, linux-api,
linux-xfs, linux-mm, iommu, Andy Lutomirski, linux-fsdevel,
Andrew Morton, Linus Torvalds, Christoph Hellwig
On Tue 10-10-17 07:49:01, Dan Williams wrote:
> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
> mechanism to define new behavior that is known to fail on older kernels
> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
> is guaranteed to fail on all legacy mmap implementations.
>
> It is worth noting that the original proposal was for a standalone
> MAP_VALIDATE flag. However, when that could not be supported by all
> archs Linus observed:
>
> I see why you *think* you want a bitmap. You think you want
> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
> etc, so that people can do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
> | MAP_SYNC, fd, 0);
>
> and "know" that MAP_SYNC actually takes.
>
> And I'm saying that whole wish is bogus. You're fundamentally
> depending on special semantics, just make it explicit. It's already
> not portable, so don't try to make it so.
>
> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
> of 0x3, and make people do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
> | MAP_SYNC, fd, 0);
>
> and then the kernel side is easier too (none of that random garbage
> playing games with looking at the "MAP_VALIDATE bit", but just another
> case statement in that map type thing.
>
> Boom. Done.
>
> Similar to ->fallocate() we also want the ability to validate the
> support for new flags on a per ->mmap() 'struct file_operations'
> instance basis. Towards that end arrange for flags to be generically
> validated against a mmap_supported_mask exported by 'struct
> file_operations'. By default all existing flags are implicitly
> supported, but new flags require MAP_SHARED_VALIDATE and
> per-instance-opt-in.
>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/alpha/include/uapi/asm/mman.h | 1 +
> arch/mips/include/uapi/asm/mman.h | 1 +
> arch/mips/kernel/vdso.c | 2 +
> arch/parisc/include/uapi/asm/mman.h | 1 +
> arch/tile/mm/elf.c | 3 +-
> arch/xtensa/include/uapi/asm/mman.h | 1 +
> include/linux/fs.h | 2 +
> include/linux/mm.h | 2 +
> include/linux/mman.h | 39 ++++++++++++++++++++++++++
> include/uapi/asm-generic/mman-common.h | 1 +
> mm/mmap.c | 21 ++++++++++++--
> tools/include/uapi/asm-generic/mman-common.h | 1 +
> 12 files changed, 69 insertions(+), 6 deletions(-)
>
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..92823f24890b 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -14,6 +14,7 @@
> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
> #define MAP_FIXED 0x100 /* Interpret addr exactly */
> #define MAP_ANONYMOUS 0x10 /* don't use a file */
> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
all archs).
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index f8c10d336e42..5c4c98e4adc9 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>
> extern unsigned long mmap_region(struct file *file, unsigned long addr,
> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
> - struct list_head *uf);
> + struct list_head *uf, unsigned long map_flags);
> extern unsigned long do_mmap(struct file *file, unsigned long addr,
> unsigned long len, unsigned long prot, unsigned long flags,
> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
I have to say I'm not very keen on passing down both vm_flags and map_flags
- vm_flags are almost a subset of map_flags but not quite and the ambiguity
which needs to be used for a particular check seems to open a space for
errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
and just pass map_flags through mmap_region() so there's no space for
confusion but future checks could do something different. But OTOH I don't
see a cleaner way of avoiding the need to allocate vma flag for something
you need to check down in ->mmap_validate so I guess I'll live with that
and if problems really happen, we may have cleaner idea what needs to be
done.
So overall feel free to add:
Reviewed-by: Jan Kara <jack@suse.cz>
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 7:43 ` Jan Kara
0 siblings, 0 replies; 77+ messages in thread
From: Jan Kara @ 2017-10-11 7:43 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw, Jan Kara, Arnd Bergmann,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Andy Lutomirski, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
Andrew Morton, Linus Torvalds, Christoph Hellwig
On Tue 10-10-17 07:49:01, Dan Williams wrote:
> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
> mechanism to define new behavior that is known to fail on older kernels
> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
> is guaranteed to fail on all legacy mmap implementations.
>
> It is worth noting that the original proposal was for a standalone
> MAP_VALIDATE flag. However, when that could not be supported by all
> archs Linus observed:
>
> I see why you *think* you want a bitmap. You think you want
> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
> etc, so that people can do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
> | MAP_SYNC, fd, 0);
>
> and "know" that MAP_SYNC actually takes.
>
> And I'm saying that whole wish is bogus. You're fundamentally
> depending on special semantics, just make it explicit. It's already
> not portable, so don't try to make it so.
>
> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
> of 0x3, and make people do
>
> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
> | MAP_SYNC, fd, 0);
>
> and then the kernel side is easier too (none of that random garbage
> playing games with looking at the "MAP_VALIDATE bit", but just another
> case statement in that map type thing.
>
> Boom. Done.
>
> Similar to ->fallocate() we also want the ability to validate the
> support for new flags on a per ->mmap() 'struct file_operations'
> instance basis. Towards that end arrange for flags to be generically
> validated against a mmap_supported_mask exported by 'struct
> file_operations'. By default all existing flags are implicitly
> supported, but new flags require MAP_SHARED_VALIDATE and
> per-instance-opt-in.
>
> Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
> Cc: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
> Cc: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
> Suggested-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> Suggested-by: Linus Torvalds <torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
> Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> ---
> arch/alpha/include/uapi/asm/mman.h | 1 +
> arch/mips/include/uapi/asm/mman.h | 1 +
> arch/mips/kernel/vdso.c | 2 +
> arch/parisc/include/uapi/asm/mman.h | 1 +
> arch/tile/mm/elf.c | 3 +-
> arch/xtensa/include/uapi/asm/mman.h | 1 +
> include/linux/fs.h | 2 +
> include/linux/mm.h | 2 +
> include/linux/mman.h | 39 ++++++++++++++++++++++++++
> include/uapi/asm-generic/mman-common.h | 1 +
> mm/mmap.c | 21 ++++++++++++--
> tools/include/uapi/asm-generic/mman-common.h | 1 +
> 12 files changed, 69 insertions(+), 6 deletions(-)
>
> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
> index 3b26cc62dadb..92823f24890b 100644
> --- a/arch/alpha/include/uapi/asm/mman.h
> +++ b/arch/alpha/include/uapi/asm/mman.h
> @@ -14,6 +14,7 @@
> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
> #define MAP_FIXED 0x100 /* Interpret addr exactly */
> #define MAP_ANONYMOUS 0x10 /* don't use a file */
> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
all archs).
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index f8c10d336e42..5c4c98e4adc9 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>
> extern unsigned long mmap_region(struct file *file, unsigned long addr,
> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
> - struct list_head *uf);
> + struct list_head *uf, unsigned long map_flags);
> extern unsigned long do_mmap(struct file *file, unsigned long addr,
> unsigned long len, unsigned long prot, unsigned long flags,
> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
I have to say I'm not very keen on passing down both vm_flags and map_flags
- vm_flags are almost a subset of map_flags but not quite and the ambiguity
which needs to be used for a particular check seems to open a space for
errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
and just pass map_flags through mmap_region() so there's no space for
confusion but future checks could do something different. But OTOH I don't
see a cleaner way of avoiding the need to allocate vma flag for something
you need to check down in ->mmap_validate so I guess I'll live with that
and if problems really happen, we may have cleaner idea what needs to be
done.
So overall feel free to add:
Reviewed-by: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Honza
--
Jan Kara <jack-IBi9RG/b67k@public.gmane.org>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 14:15 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 14:15 UTC (permalink / raw)
To: Jan Kara
Cc: Arnd Bergmann, linux-nvdimm, linux-rdma, Linux API, linux-xfs,
Linux MM, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
On Wed, Oct 11, 2017 at 12:43 AM, Jan Kara <jack@suse.cz> wrote:
> On Tue 10-10-17 07:49:01, Dan Williams wrote:
>> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
>> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
>> mechanism to define new behavior that is known to fail on older kernels
>> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
>> is guaranteed to fail on all legacy mmap implementations.
>>
>> It is worth noting that the original proposal was for a standalone
>> MAP_VALIDATE flag. However, when that could not be supported by all
>> archs Linus observed:
>>
>> I see why you *think* you want a bitmap. You think you want
>> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
>> etc, so that people can do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
>> | MAP_SYNC, fd, 0);
>>
>> and "know" that MAP_SYNC actually takes.
>>
>> And I'm saying that whole wish is bogus. You're fundamentally
>> depending on special semantics, just make it explicit. It's already
>> not portable, so don't try to make it so.
>>
>> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
>> of 0x3, and make people do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
>> | MAP_SYNC, fd, 0);
>>
>> and then the kernel side is easier too (none of that random garbage
>> playing games with looking at the "MAP_VALIDATE bit", but just another
>> case statement in that map type thing.
>>
>> Boom. Done.
>>
>> Similar to ->fallocate() we also want the ability to validate the
>> support for new flags on a per ->mmap() 'struct file_operations'
>> instance basis. Towards that end arrange for flags to be generically
>> validated against a mmap_supported_mask exported by 'struct
>> file_operations'. By default all existing flags are implicitly
>> supported, but new flags require MAP_SHARED_VALIDATE and
>> per-instance-opt-in.
>>
>> Cc: Jan Kara <jack@suse.cz>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: Andy Lutomirski <luto@kernel.org>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Suggested-by: Christoph Hellwig <hch@lst.de>
>> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>> ---
>> arch/alpha/include/uapi/asm/mman.h | 1 +
>> arch/mips/include/uapi/asm/mman.h | 1 +
>> arch/mips/kernel/vdso.c | 2 +
>> arch/parisc/include/uapi/asm/mman.h | 1 +
>> arch/tile/mm/elf.c | 3 +-
>> arch/xtensa/include/uapi/asm/mman.h | 1 +
>> include/linux/fs.h | 2 +
>> include/linux/mm.h | 2 +
>> include/linux/mman.h | 39 ++++++++++++++++++++++++++
>> include/uapi/asm-generic/mman-common.h | 1 +
>> mm/mmap.c | 21 ++++++++++++--
>> tools/include/uapi/asm-generic/mman-common.h | 1 +
>> 12 files changed, 69 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
>> index 3b26cc62dadb..92823f24890b 100644
>> --- a/arch/alpha/include/uapi/asm/mman.h
>> +++ b/arch/alpha/include/uapi/asm/mman.h
>> @@ -14,6 +14,7 @@
>> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
>> #define MAP_FIXED 0x100 /* Interpret addr exactly */
>> #define MAP_ANONYMOUS 0x10 /* don't use a file */
>> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
>
> Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
> definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
> all archs).
Will do.
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index f8c10d336e42..5c4c98e4adc9 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>>
>> extern unsigned long mmap_region(struct file *file, unsigned long addr,
>> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
>> - struct list_head *uf);
>> + struct list_head *uf, unsigned long map_flags);
>> extern unsigned long do_mmap(struct file *file, unsigned long addr,
>> unsigned long len, unsigned long prot, unsigned long flags,
>> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
>
> I have to say I'm not very keen on passing down both vm_flags and map_flags
> - vm_flags are almost a subset of map_flags but not quite and the ambiguity
> which needs to be used for a particular check seems to open a space for
> errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
> and just pass map_flags through mmap_region() so there's no space for
> confusion but future checks could do something different.
I was hoping the fact that one can't trigger a call to
->mmap_validate() unless they specify a flag outside of
LEGACY_MAP_MASK makes it clearer that validation is only for new
flags. Old flags get the existing "may be silently ignored" behavior.
> But OTOH I don't
> see a cleaner way of avoiding the need to allocate vma flag for something
> you need to check down in ->mmap_validate so I guess I'll live with that
> and if problems really happen, we may have cleaner idea what needs to be
> done.
>
> So overall feel free to add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
Thanks Jan.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 14:15 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 14:15 UTC (permalink / raw)
To: Jan Kara
Cc: linux-nvdimm, Arnd Bergmann, linux-rdma, Linux API, linux-xfs,
Linux MM, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
On Wed, Oct 11, 2017 at 12:43 AM, Jan Kara <jack@suse.cz> wrote:
> On Tue 10-10-17 07:49:01, Dan Williams wrote:
>> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
>> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
>> mechanism to define new behavior that is known to fail on older kernels
>> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
>> is guaranteed to fail on all legacy mmap implementations.
>>
>> It is worth noting that the original proposal was for a standalone
>> MAP_VALIDATE flag. However, when that could not be supported by all
>> archs Linus observed:
>>
>> I see why you *think* you want a bitmap. You think you want
>> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
>> etc, so that people can do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
>> | MAP_SYNC, fd, 0);
>>
>> and "know" that MAP_SYNC actually takes.
>>
>> And I'm saying that whole wish is bogus. You're fundamentally
>> depending on special semantics, just make it explicit. It's already
>> not portable, so don't try to make it so.
>>
>> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
>> of 0x3, and make people do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
>> | MAP_SYNC, fd, 0);
>>
>> and then the kernel side is easier too (none of that random garbage
>> playing games with looking at the "MAP_VALIDATE bit", but just another
>> case statement in that map type thing.
>>
>> Boom. Done.
>>
>> Similar to ->fallocate() we also want the ability to validate the
>> support for new flags on a per ->mmap() 'struct file_operations'
>> instance basis. Towards that end arrange for flags to be generically
>> validated against a mmap_supported_mask exported by 'struct
>> file_operations'. By default all existing flags are implicitly
>> supported, but new flags require MAP_SHARED_VALIDATE and
>> per-instance-opt-in.
>>
>> Cc: Jan Kara <jack@suse.cz>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: Andy Lutomirski <luto@kernel.org>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Suggested-by: Christoph Hellwig <hch@lst.de>
>> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>> ---
>> arch/alpha/include/uapi/asm/mman.h | 1 +
>> arch/mips/include/uapi/asm/mman.h | 1 +
>> arch/mips/kernel/vdso.c | 2 +
>> arch/parisc/include/uapi/asm/mman.h | 1 +
>> arch/tile/mm/elf.c | 3 +-
>> arch/xtensa/include/uapi/asm/mman.h | 1 +
>> include/linux/fs.h | 2 +
>> include/linux/mm.h | 2 +
>> include/linux/mman.h | 39 ++++++++++++++++++++++++++
>> include/uapi/asm-generic/mman-common.h | 1 +
>> mm/mmap.c | 21 ++++++++++++--
>> tools/include/uapi/asm-generic/mman-common.h | 1 +
>> 12 files changed, 69 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
>> index 3b26cc62dadb..92823f24890b 100644
>> --- a/arch/alpha/include/uapi/asm/mman.h
>> +++ b/arch/alpha/include/uapi/asm/mman.h
>> @@ -14,6 +14,7 @@
>> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
>> #define MAP_FIXED 0x100 /* Interpret addr exactly */
>> #define MAP_ANONYMOUS 0x10 /* don't use a file */
>> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
>
> Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
> definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
> all archs).
Will do.
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index f8c10d336e42..5c4c98e4adc9 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>>
>> extern unsigned long mmap_region(struct file *file, unsigned long addr,
>> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
>> - struct list_head *uf);
>> + struct list_head *uf, unsigned long map_flags);
>> extern unsigned long do_mmap(struct file *file, unsigned long addr,
>> unsigned long len, unsigned long prot, unsigned long flags,
>> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
>
> I have to say I'm not very keen on passing down both vm_flags and map_flags
> - vm_flags are almost a subset of map_flags but not quite and the ambiguity
> which needs to be used for a particular check seems to open a space for
> errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
> and just pass map_flags through mmap_region() so there's no space for
> confusion but future checks could do something different.
I was hoping the fact that one can't trigger a call to
->mmap_validate() unless they specify a flag outside of
LEGACY_MAP_MASK makes it clearer that validation is only for new
flags. Old flags get the existing "may be silently ignored" behavior.
> But OTOH I don't
> see a cleaner way of avoiding the need to allocate vma flag for something
> you need to check down in ->mmap_validate so I guess I'll live with that
> and if problems really happen, we may have cleaner idea what needs to be
> done.
>
> So overall feel free to add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
Thanks Jan.
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 14:15 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 14:15 UTC (permalink / raw)
To: Jan Kara
Cc: linux-nvdimm, Arnd Bergmann, linux-rdma, Linux API, linux-xfs,
Linux MM, iommu, Andy Lutomirski, linux-fsdevel, Andrew Morton,
Linus Torvalds, Christoph Hellwig
On Wed, Oct 11, 2017 at 12:43 AM, Jan Kara <jack@suse.cz> wrote:
> On Tue 10-10-17 07:49:01, Dan Williams wrote:
>> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
>> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
>> mechanism to define new behavior that is known to fail on older kernels
>> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
>> is guaranteed to fail on all legacy mmap implementations.
>>
>> It is worth noting that the original proposal was for a standalone
>> MAP_VALIDATE flag. However, when that could not be supported by all
>> archs Linus observed:
>>
>> I see why you *think* you want a bitmap. You think you want
>> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
>> etc, so that people can do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
>> | MAP_SYNC, fd, 0);
>>
>> and "know" that MAP_SYNC actually takes.
>>
>> And I'm saying that whole wish is bogus. You're fundamentally
>> depending on special semantics, just make it explicit. It's already
>> not portable, so don't try to make it so.
>>
>> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
>> of 0x3, and make people do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
>> | MAP_SYNC, fd, 0);
>>
>> and then the kernel side is easier too (none of that random garbage
>> playing games with looking at the "MAP_VALIDATE bit", but just another
>> case statement in that map type thing.
>>
>> Boom. Done.
>>
>> Similar to ->fallocate() we also want the ability to validate the
>> support for new flags on a per ->mmap() 'struct file_operations'
>> instance basis. Towards that end arrange for flags to be generically
>> validated against a mmap_supported_mask exported by 'struct
>> file_operations'. By default all existing flags are implicitly
>> supported, but new flags require MAP_SHARED_VALIDATE and
>> per-instance-opt-in.
>>
>> Cc: Jan Kara <jack@suse.cz>
>> Cc: Arnd Bergmann <arnd@arndb.de>
>> Cc: Andy Lutomirski <luto@kernel.org>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Suggested-by: Christoph Hellwig <hch@lst.de>
>> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
>> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
>> ---
>> arch/alpha/include/uapi/asm/mman.h | 1 +
>> arch/mips/include/uapi/asm/mman.h | 1 +
>> arch/mips/kernel/vdso.c | 2 +
>> arch/parisc/include/uapi/asm/mman.h | 1 +
>> arch/tile/mm/elf.c | 3 +-
>> arch/xtensa/include/uapi/asm/mman.h | 1 +
>> include/linux/fs.h | 2 +
>> include/linux/mm.h | 2 +
>> include/linux/mman.h | 39 ++++++++++++++++++++++++++
>> include/uapi/asm-generic/mman-common.h | 1 +
>> mm/mmap.c | 21 ++++++++++++--
>> tools/include/uapi/asm-generic/mman-common.h | 1 +
>> 12 files changed, 69 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
>> index 3b26cc62dadb..92823f24890b 100644
>> --- a/arch/alpha/include/uapi/asm/mman.h
>> +++ b/arch/alpha/include/uapi/asm/mman.h
>> @@ -14,6 +14,7 @@
>> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
>> #define MAP_FIXED 0x100 /* Interpret addr exactly */
>> #define MAP_ANONYMOUS 0x10 /* don't use a file */
>> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
>
> Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
> definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
> all archs).
Will do.
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index f8c10d336e42..5c4c98e4adc9 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>>
>> extern unsigned long mmap_region(struct file *file, unsigned long addr,
>> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
>> - struct list_head *uf);
>> + struct list_head *uf, unsigned long map_flags);
>> extern unsigned long do_mmap(struct file *file, unsigned long addr,
>> unsigned long len, unsigned long prot, unsigned long flags,
>> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
>
> I have to say I'm not very keen on passing down both vm_flags and map_flags
> - vm_flags are almost a subset of map_flags but not quite and the ambiguity
> which needs to be used for a particular check seems to open a space for
> errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
> and just pass map_flags through mmap_region() so there's no space for
> confusion but future checks could do something different.
I was hoping the fact that one can't trigger a call to
->mmap_validate() unless they specify a flag outside of
LEGACY_MAP_MASK makes it clearer that validation is only for new
flags. Old flags get the existing "may be silently ignored" behavior.
> But OTOH I don't
> see a cleaner way of avoiding the need to allocate vma flag for something
> you need to check down in ->mmap_validate so I guess I'll live with that
> and if problems really happen, we may have cleaner idea what needs to be
> done.
>
> So overall feel free to add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
Thanks Jan.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/14] mm: introduce MAP_SHARED_VALIDATE, a mechanism to safely define new mmap flags
@ 2017-10-11 14:15 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 14:15 UTC (permalink / raw)
To: Jan Kara
Cc: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw, Arnd Bergmann,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Linux API,
linux-xfs-u79uwXL29TY76Z2rM5mHXA, Linux MM,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Andy Lutomirski, linux-fsdevel, Andrew Morton, Linus Torvalds,
Christoph Hellwig
On Wed, Oct 11, 2017 at 12:43 AM, Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org> wrote:
> On Tue 10-10-17 07:49:01, Dan Williams wrote:
>> The mmap(2) syscall suffers from the ABI anti-pattern of not validating
>> unknown flags. However, proposals like MAP_SYNC and MAP_DIRECT need a
>> mechanism to define new behavior that is known to fail on older kernels
>> without the support. Define a new MAP_SHARED_VALIDATE flag pattern that
>> is guaranteed to fail on all legacy mmap implementations.
>>
>> It is worth noting that the original proposal was for a standalone
>> MAP_VALIDATE flag. However, when that could not be supported by all
>> archs Linus observed:
>>
>> I see why you *think* you want a bitmap. You think you want
>> a bitmap because you want to make MAP_VALIDATE be part of MAP_SYNC
>> etc, so that people can do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED
>> | MAP_SYNC, fd, 0);
>>
>> and "know" that MAP_SYNC actually takes.
>>
>> And I'm saying that whole wish is bogus. You're fundamentally
>> depending on special semantics, just make it explicit. It's already
>> not portable, so don't try to make it so.
>>
>> Rename that MAP_VALIDATE as MAP_SHARED_VALIDATE, make it have a value
>> of 0x3, and make people do
>>
>> ret = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED_VALIDATE
>> | MAP_SYNC, fd, 0);
>>
>> and then the kernel side is easier too (none of that random garbage
>> playing games with looking at the "MAP_VALIDATE bit", but just another
>> case statement in that map type thing.
>>
>> Boom. Done.
>>
>> Similar to ->fallocate() we also want the ability to validate the
>> support for new flags on a per ->mmap() 'struct file_operations'
>> instance basis. Towards that end arrange for flags to be generically
>> validated against a mmap_supported_mask exported by 'struct
>> file_operations'. By default all existing flags are implicitly
>> supported, but new flags require MAP_SHARED_VALIDATE and
>> per-instance-opt-in.
>>
>> Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
>> Cc: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
>> Cc: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
>> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
>> Suggested-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
>> Suggested-by: Linus Torvalds <torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
>> Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> ---
>> arch/alpha/include/uapi/asm/mman.h | 1 +
>> arch/mips/include/uapi/asm/mman.h | 1 +
>> arch/mips/kernel/vdso.c | 2 +
>> arch/parisc/include/uapi/asm/mman.h | 1 +
>> arch/tile/mm/elf.c | 3 +-
>> arch/xtensa/include/uapi/asm/mman.h | 1 +
>> include/linux/fs.h | 2 +
>> include/linux/mm.h | 2 +
>> include/linux/mman.h | 39 ++++++++++++++++++++++++++
>> include/uapi/asm-generic/mman-common.h | 1 +
>> mm/mmap.c | 21 ++++++++++++--
>> tools/include/uapi/asm-generic/mman-common.h | 1 +
>> 12 files changed, 69 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
>> index 3b26cc62dadb..92823f24890b 100644
>> --- a/arch/alpha/include/uapi/asm/mman.h
>> +++ b/arch/alpha/include/uapi/asm/mman.h
>> @@ -14,6 +14,7 @@
>> #define MAP_TYPE 0x0f /* Mask for type of mapping (OSF/1 is _wrong_) */
>> #define MAP_FIXED 0x100 /* Interpret addr exactly */
>> #define MAP_ANONYMOUS 0x10 /* don't use a file */
>> +#define MAP_SHARED_VALIDATE 0x3 /* share + validate extension flags */
>
> Just a nit but I'd put definition of MAP_SHARED_VALIDATE close to the
> definition of MAP_SHARED and MAP_PRIVATE where it logically belongs (for
> all archs).
Will do.
>
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index f8c10d336e42..5c4c98e4adc9 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -2133,7 +2133,7 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
>>
>> extern unsigned long mmap_region(struct file *file, unsigned long addr,
>> unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
>> - struct list_head *uf);
>> + struct list_head *uf, unsigned long map_flags);
>> extern unsigned long do_mmap(struct file *file, unsigned long addr,
>> unsigned long len, unsigned long prot, unsigned long flags,
>> vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
>
> I have to say I'm not very keen on passing down both vm_flags and map_flags
> - vm_flags are almost a subset of map_flags but not quite and the ambiguity
> which needs to be used for a particular check seems to open a space for
> errors. Granted you currently only care about MAP_DIRECT in ->mmap_validate
> and just pass map_flags through mmap_region() so there's no space for
> confusion but future checks could do something different.
I was hoping the fact that one can't trigger a call to
->mmap_validate() unless they specify a flag outside of
LEGACY_MAP_MASK makes it clearer that validation is only for new
flags. Old flags get the existing "may be silently ignored" behavior.
> But OTOH I don't
> see a cleaner way of avoiding the need to allocate vma flag for something
> you need to check down in ->mmap_validate so I guess I'll live with that
> and if problems really happen, we may have cleaner idea what needs to be
> done.
>
> So overall feel free to add:
>
> Reviewed-by: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Thanks Jan.
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 02/14] fs, mm: pass fd to ->mmap_validate()
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:49 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
linux-xfs, linux-mm, iommu, linux-fsdevel, Andrew Morton,
Christoph Hellwig
The MAP_DIRECT mechanism for mmap intends to use a file lease to prevent
block map changes while the file is mapped. It requires the fd to setup
an fasync_struct for signalling lease break events to the lease holder.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/mips/kernel/vdso.c | 2 +-
arch/tile/mm/elf.c | 2 +-
arch/x86/mm/mpx.c | 3 ++-
fs/aio.c | 2 +-
include/linux/fs.h | 2 +-
include/linux/mm.h | 9 +++++----
ipc/shm.c | 3 ++-
mm/internal.h | 2 +-
mm/mmap.c | 13 +++++++------
mm/nommu.c | 5 +++--
mm/util.c | 7 ++++---
11 files changed, 28 insertions(+), 22 deletions(-)
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index cf10654477a9..ab26c7ac0316 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL, 0);
+ 0, NULL, 0, -1);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 5ffcbe76aef9..61a9588e141a 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -144,7 +144,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
- NULL, 0);
+ NULL, 0, -1);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index 9ceaa955d2ba..a8baa94a496b 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -52,7 +52,8 @@ static unsigned long mpx_mmap(unsigned long len)
down_write(&mm->mmap_sem);
addr = do_mmap(NULL, 0, len, PROT_READ | PROT_WRITE,
- MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate, NULL);
+ MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate,
+ NULL, -1);
up_write(&mm->mmap_sem);
if (populate)
mm_populate(addr, populate);
diff --git a/fs/aio.c b/fs/aio.c
index 5a2487217072..d10ca6db2ee6 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -519,7 +519,7 @@ static int aio_setup_ring(struct kioctx *ctx, unsigned int nr_events)
ctx->mmap_base = do_mmap_pgoff(ctx->aio_ring_file, 0, ctx->mmap_size,
PROT_READ | PROT_WRITE,
- MAP_SHARED, 0, &unused, NULL);
+ MAP_SHARED, 0, &unused, NULL, -1);
up_write(&mm->mmap_sem);
if (IS_ERR((void *)ctx->mmap_base)) {
ctx->mmap_size = 0;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 51538958f7f5..c2b9bf3dc4e9 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1702,7 +1702,7 @@ struct file_operations {
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*mmap_validate) (struct file *, struct vm_area_struct *,
- unsigned long);
+ unsigned long, int);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 5c4c98e4adc9..0afa19feb755 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,11 +2133,11 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags);
+ struct list_head *uf, unsigned long map_flags, int fd);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
- struct list_head *uf);
+ struct list_head *uf, int fd);
extern int do_munmap(struct mm_struct *, unsigned long, size_t,
struct list_head *uf);
@@ -2145,9 +2145,10 @@ static inline unsigned long
do_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
- return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate, uf);
+ return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate,
+ uf, fd);
}
#ifdef CONFIG_MMU
diff --git a/ipc/shm.c b/ipc/shm.c
index 1e2b1692ba2c..585e05eef40a 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -1399,7 +1399,8 @@ long do_shmat(int shmid, char __user *shmaddr, int shmflg,
goto invalid;
}
- addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate, NULL);
+ addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate,
+ NULL, -1);
*raddr = addr;
err = 0;
if (IS_ERR_VALUE(addr))
diff --git a/mm/internal.h b/mm/internal.h
index 1df011f62480..70ed7b06dd85 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -466,7 +466,7 @@ extern u32 hwpoison_filter_enable;
extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long,
unsigned long, unsigned long,
- unsigned long, unsigned long);
+ unsigned long, unsigned long, int);
extern void set_pageblock_order(void);
unsigned long reclaim_clean_pages_from_list(struct zone *zone,
diff --git a/mm/mmap.c b/mm/mmap.c
index a1bcaa9eff42..c2cb6334a7a9 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1322,7 +1322,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
unsigned long flags, vm_flags_t vm_flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
struct mm_struct *mm = current->mm;
int pkey = 0;
@@ -1477,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags, fd);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1527,7 +1527,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
out_fput:
if (file)
fput(file);
@@ -1614,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags)
+ struct list_head *uf, unsigned long map_flags, int fd)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1700,7 +1700,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
*/
vma->vm_file = get_file(file);
if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
- error = file->f_op->mmap_validate(file, vma, map_flags);
+ error = file->f_op->mmap_validate(file, vma,
+ map_flags, fd);
else
error = call_mmap(file, vma);
if (error)
@@ -2842,7 +2843,7 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
file = get_file(vma->vm_file);
ret = do_mmap_pgoff(vma->vm_file, start, size,
- prot, flags, pgoff, &populate, NULL);
+ prot, flags, pgoff, &populate, NULL, -1);
fput(file);
out:
up_write(&mm->mmap_sem);
diff --git a/mm/nommu.c b/mm/nommu.c
index 17c00d93de2e..952d205d3b66 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1206,7 +1206,8 @@ unsigned long do_mmap(struct file *file,
vm_flags_t vm_flags,
unsigned long pgoff,
unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf,
+ int fd)
{
struct vm_area_struct *vma;
struct vm_region *region;
@@ -1439,7 +1440,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
if (file)
fput(file);
diff --git a/mm/util.c b/mm/util.c
index 34e57fae959d..dcf48d929185 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -319,7 +319,7 @@ EXPORT_SYMBOL_GPL(get_user_pages_fast);
unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
- unsigned long flag, unsigned long pgoff)
+ unsigned long flag, unsigned long pgoff, int fd)
{
unsigned long ret;
struct mm_struct *mm = current->mm;
@@ -331,7 +331,7 @@ unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
ret = do_mmap_pgoff(file, addr, len, prot, flag, pgoff,
- &populate, &uf);
+ &populate, &uf, fd);
up_write(&mm->mmap_sem);
userfaultfd_unmap_complete(mm, &uf);
if (populate)
@@ -349,7 +349,8 @@ unsigned long vm_mmap(struct file *file, unsigned long addr,
if (unlikely(offset_in_page(offset)))
return -EINVAL;
- return vm_mmap_pgoff(file, addr, len, prot, flag, offset >> PAGE_SHIFT);
+ return vm_mmap_pgoff(file, addr, len, prot, flag,
+ offset >> PAGE_SHIFT, -1);
}
EXPORT_SYMBOL(vm_mmap);
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 02/14] fs, mm: pass fd to ->mmap_validate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, Christoph Hellwig, linux-xfs, linux-mm, Jeff Moyer,
linux-fsdevel, Andrew Morton, Ross Zwisler
The MAP_DIRECT mechanism for mmap intends to use a file lease to prevent
block map changes while the file is mapped. It requires the fd to setup
an fasync_struct for signalling lease break events to the lease holder.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/mips/kernel/vdso.c | 2 +-
arch/tile/mm/elf.c | 2 +-
arch/x86/mm/mpx.c | 3 ++-
fs/aio.c | 2 +-
include/linux/fs.h | 2 +-
include/linux/mm.h | 9 +++++----
ipc/shm.c | 3 ++-
mm/internal.h | 2 +-
mm/mmap.c | 13 +++++++------
mm/nommu.c | 5 +++--
mm/util.c | 7 ++++---
11 files changed, 28 insertions(+), 22 deletions(-)
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index cf10654477a9..ab26c7ac0316 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL, 0);
+ 0, NULL, 0, -1);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 5ffcbe76aef9..61a9588e141a 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -144,7 +144,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
- NULL, 0);
+ NULL, 0, -1);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index 9ceaa955d2ba..a8baa94a496b 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -52,7 +52,8 @@ static unsigned long mpx_mmap(unsigned long len)
down_write(&mm->mmap_sem);
addr = do_mmap(NULL, 0, len, PROT_READ | PROT_WRITE,
- MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate, NULL);
+ MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate,
+ NULL, -1);
up_write(&mm->mmap_sem);
if (populate)
mm_populate(addr, populate);
diff --git a/fs/aio.c b/fs/aio.c
index 5a2487217072..d10ca6db2ee6 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -519,7 +519,7 @@ static int aio_setup_ring(struct kioctx *ctx, unsigned int nr_events)
ctx->mmap_base = do_mmap_pgoff(ctx->aio_ring_file, 0, ctx->mmap_size,
PROT_READ | PROT_WRITE,
- MAP_SHARED, 0, &unused, NULL);
+ MAP_SHARED, 0, &unused, NULL, -1);
up_write(&mm->mmap_sem);
if (IS_ERR((void *)ctx->mmap_base)) {
ctx->mmap_size = 0;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 51538958f7f5..c2b9bf3dc4e9 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1702,7 +1702,7 @@ struct file_operations {
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*mmap_validate) (struct file *, struct vm_area_struct *,
- unsigned long);
+ unsigned long, int);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 5c4c98e4adc9..0afa19feb755 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,11 +2133,11 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags);
+ struct list_head *uf, unsigned long map_flags, int fd);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
- struct list_head *uf);
+ struct list_head *uf, int fd);
extern int do_munmap(struct mm_struct *, unsigned long, size_t,
struct list_head *uf);
@@ -2145,9 +2145,10 @@ static inline unsigned long
do_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
- return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate, uf);
+ return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate,
+ uf, fd);
}
#ifdef CONFIG_MMU
diff --git a/ipc/shm.c b/ipc/shm.c
index 1e2b1692ba2c..585e05eef40a 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -1399,7 +1399,8 @@ long do_shmat(int shmid, char __user *shmaddr, int shmflg,
goto invalid;
}
- addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate, NULL);
+ addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate,
+ NULL, -1);
*raddr = addr;
err = 0;
if (IS_ERR_VALUE(addr))
diff --git a/mm/internal.h b/mm/internal.h
index 1df011f62480..70ed7b06dd85 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -466,7 +466,7 @@ extern u32 hwpoison_filter_enable;
extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long,
unsigned long, unsigned long,
- unsigned long, unsigned long);
+ unsigned long, unsigned long, int);
extern void set_pageblock_order(void);
unsigned long reclaim_clean_pages_from_list(struct zone *zone,
diff --git a/mm/mmap.c b/mm/mmap.c
index a1bcaa9eff42..c2cb6334a7a9 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1322,7 +1322,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
unsigned long flags, vm_flags_t vm_flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
struct mm_struct *mm = current->mm;
int pkey = 0;
@@ -1477,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags, fd);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1527,7 +1527,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
out_fput:
if (file)
fput(file);
@@ -1614,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags)
+ struct list_head *uf, unsigned long map_flags, int fd)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1700,7 +1700,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
*/
vma->vm_file = get_file(file);
if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
- error = file->f_op->mmap_validate(file, vma, map_flags);
+ error = file->f_op->mmap_validate(file, vma,
+ map_flags, fd);
else
error = call_mmap(file, vma);
if (error)
@@ -2842,7 +2843,7 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
file = get_file(vma->vm_file);
ret = do_mmap_pgoff(vma->vm_file, start, size,
- prot, flags, pgoff, &populate, NULL);
+ prot, flags, pgoff, &populate, NULL, -1);
fput(file);
out:
up_write(&mm->mmap_sem);
diff --git a/mm/nommu.c b/mm/nommu.c
index 17c00d93de2e..952d205d3b66 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1206,7 +1206,8 @@ unsigned long do_mmap(struct file *file,
vm_flags_t vm_flags,
unsigned long pgoff,
unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf,
+ int fd)
{
struct vm_area_struct *vma;
struct vm_region *region;
@@ -1439,7 +1440,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
if (file)
fput(file);
diff --git a/mm/util.c b/mm/util.c
index 34e57fae959d..dcf48d929185 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -319,7 +319,7 @@ EXPORT_SYMBOL_GPL(get_user_pages_fast);
unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
- unsigned long flag, unsigned long pgoff)
+ unsigned long flag, unsigned long pgoff, int fd)
{
unsigned long ret;
struct mm_struct *mm = current->mm;
@@ -331,7 +331,7 @@ unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
ret = do_mmap_pgoff(file, addr, len, prot, flag, pgoff,
- &populate, &uf);
+ &populate, &uf, fd);
up_write(&mm->mmap_sem);
userfaultfd_unmap_complete(mm, &uf);
if (populate)
@@ -349,7 +349,8 @@ unsigned long vm_mmap(struct file *file, unsigned long addr,
if (unlikely(offset_in_page(offset)))
return -EINVAL;
- return vm_mmap_pgoff(file, addr, len, prot, flag, offset >> PAGE_SHIFT);
+ return vm_mmap_pgoff(file, addr, len, prot, flag,
+ offset >> PAGE_SHIFT, -1);
}
EXPORT_SYMBOL(vm_mmap);
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 02/14] fs, mm: pass fd to ->mmap_validate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, Christoph Hellwig, linux-xfs, linux-mm, Jeff Moyer,
linux-fsdevel, Andrew Morton, Ross Zwisler
The MAP_DIRECT mechanism for mmap intends to use a file lease to prevent
block map changes while the file is mapped. It requires the fd to setup
an fasync_struct for signalling lease break events to the lease holder.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/mips/kernel/vdso.c | 2 +-
arch/tile/mm/elf.c | 2 +-
arch/x86/mm/mpx.c | 3 ++-
fs/aio.c | 2 +-
include/linux/fs.h | 2 +-
include/linux/mm.h | 9 +++++----
ipc/shm.c | 3 ++-
mm/internal.h | 2 +-
mm/mmap.c | 13 +++++++------
mm/nommu.c | 5 +++--
mm/util.c | 7 ++++---
11 files changed, 28 insertions(+), 22 deletions(-)
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index cf10654477a9..ab26c7ac0316 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -110,7 +110,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
base = mmap_region(NULL, STACK_TOP, PAGE_SIZE,
VM_READ|VM_WRITE|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- 0, NULL, 0);
+ 0, NULL, 0, -1);
if (IS_ERR_VALUE(base)) {
ret = base;
goto out;
diff --git a/arch/tile/mm/elf.c b/arch/tile/mm/elf.c
index 5ffcbe76aef9..61a9588e141a 100644
--- a/arch/tile/mm/elf.c
+++ b/arch/tile/mm/elf.c
@@ -144,7 +144,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
addr = mmap_region(NULL, addr, INTRPT_SIZE,
VM_READ|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC, 0,
- NULL, 0);
+ NULL, 0, -1);
if (addr > (unsigned long) -PAGE_SIZE)
retval = (int) addr;
}
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index 9ceaa955d2ba..a8baa94a496b 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -52,7 +52,8 @@ static unsigned long mpx_mmap(unsigned long len)
down_write(&mm->mmap_sem);
addr = do_mmap(NULL, 0, len, PROT_READ | PROT_WRITE,
- MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate, NULL);
+ MAP_ANONYMOUS | MAP_PRIVATE, VM_MPX, 0, &populate,
+ NULL, -1);
up_write(&mm->mmap_sem);
if (populate)
mm_populate(addr, populate);
diff --git a/fs/aio.c b/fs/aio.c
index 5a2487217072..d10ca6db2ee6 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -519,7 +519,7 @@ static int aio_setup_ring(struct kioctx *ctx, unsigned int nr_events)
ctx->mmap_base = do_mmap_pgoff(ctx->aio_ring_file, 0, ctx->mmap_size,
PROT_READ | PROT_WRITE,
- MAP_SHARED, 0, &unused, NULL);
+ MAP_SHARED, 0, &unused, NULL, -1);
up_write(&mm->mmap_sem);
if (IS_ERR((void *)ctx->mmap_base)) {
ctx->mmap_size = 0;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 51538958f7f5..c2b9bf3dc4e9 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1702,7 +1702,7 @@ struct file_operations {
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*mmap_validate) (struct file *, struct vm_area_struct *,
- unsigned long);
+ unsigned long, int);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 5c4c98e4adc9..0afa19feb755 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2133,11 +2133,11 @@ extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned lo
extern unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags);
+ struct list_head *uf, unsigned long map_flags, int fd);
extern unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
vm_flags_t vm_flags, unsigned long pgoff, unsigned long *populate,
- struct list_head *uf);
+ struct list_head *uf, int fd);
extern int do_munmap(struct mm_struct *, unsigned long, size_t,
struct list_head *uf);
@@ -2145,9 +2145,10 @@ static inline unsigned long
do_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot, unsigned long flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
- return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate, uf);
+ return do_mmap(file, addr, len, prot, flags, 0, pgoff, populate,
+ uf, fd);
}
#ifdef CONFIG_MMU
diff --git a/ipc/shm.c b/ipc/shm.c
index 1e2b1692ba2c..585e05eef40a 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -1399,7 +1399,8 @@ long do_shmat(int shmid, char __user *shmaddr, int shmflg,
goto invalid;
}
- addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate, NULL);
+ addr = do_mmap_pgoff(file, addr, size, prot, flags, 0, &populate,
+ NULL, -1);
*raddr = addr;
err = 0;
if (IS_ERR_VALUE(addr))
diff --git a/mm/internal.h b/mm/internal.h
index 1df011f62480..70ed7b06dd85 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -466,7 +466,7 @@ extern u32 hwpoison_filter_enable;
extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long,
unsigned long, unsigned long,
- unsigned long, unsigned long);
+ unsigned long, unsigned long, int);
extern void set_pageblock_order(void);
unsigned long reclaim_clean_pages_from_list(struct zone *zone,
diff --git a/mm/mmap.c b/mm/mmap.c
index a1bcaa9eff42..c2cb6334a7a9 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1322,7 +1322,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
unsigned long flags, vm_flags_t vm_flags,
unsigned long pgoff, unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf, int fd)
{
struct mm_struct *mm = current->mm;
int pkey = 0;
@@ -1477,7 +1477,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr,
vm_flags |= VM_NORESERVE;
}
- addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags);
+ addr = mmap_region(file, addr, len, vm_flags, pgoff, uf, flags, fd);
if (!IS_ERR_VALUE(addr) &&
((vm_flags & VM_LOCKED) ||
(flags & (MAP_POPULATE | MAP_NONBLOCK)) == MAP_POPULATE))
@@ -1527,7 +1527,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
out_fput:
if (file)
fput(file);
@@ -1614,7 +1614,7 @@ static inline int accountable_mapping(struct file *file, vm_flags_t vm_flags)
unsigned long mmap_region(struct file *file, unsigned long addr,
unsigned long len, vm_flags_t vm_flags, unsigned long pgoff,
- struct list_head *uf, unsigned long map_flags)
+ struct list_head *uf, unsigned long map_flags, int fd)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev;
@@ -1700,7 +1700,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
*/
vma->vm_file = get_file(file);
if ((map_flags & MAP_TYPE) == MAP_SHARED_VALIDATE)
- error = file->f_op->mmap_validate(file, vma, map_flags);
+ error = file->f_op->mmap_validate(file, vma,
+ map_flags, fd);
else
error = call_mmap(file, vma);
if (error)
@@ -2842,7 +2843,7 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size,
file = get_file(vma->vm_file);
ret = do_mmap_pgoff(vma->vm_file, start, size,
- prot, flags, pgoff, &populate, NULL);
+ prot, flags, pgoff, &populate, NULL, -1);
fput(file);
out:
up_write(&mm->mmap_sem);
diff --git a/mm/nommu.c b/mm/nommu.c
index 17c00d93de2e..952d205d3b66 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1206,7 +1206,8 @@ unsigned long do_mmap(struct file *file,
vm_flags_t vm_flags,
unsigned long pgoff,
unsigned long *populate,
- struct list_head *uf)
+ struct list_head *uf,
+ int fd)
{
struct vm_area_struct *vma;
struct vm_region *region;
@@ -1439,7 +1440,7 @@ SYSCALL_DEFINE6(mmap_pgoff, unsigned long, addr, unsigned long, len,
flags &= ~(MAP_EXECUTABLE | MAP_DENYWRITE);
- retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff);
+ retval = vm_mmap_pgoff(file, addr, len, prot, flags, pgoff, fd);
if (file)
fput(file);
diff --git a/mm/util.c b/mm/util.c
index 34e57fae959d..dcf48d929185 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -319,7 +319,7 @@ EXPORT_SYMBOL_GPL(get_user_pages_fast);
unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
- unsigned long flag, unsigned long pgoff)
+ unsigned long flag, unsigned long pgoff, int fd)
{
unsigned long ret;
struct mm_struct *mm = current->mm;
@@ -331,7 +331,7 @@ unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
ret = do_mmap_pgoff(file, addr, len, prot, flag, pgoff,
- &populate, &uf);
+ &populate, &uf, fd);
up_write(&mm->mmap_sem);
userfaultfd_unmap_complete(mm, &uf);
if (populate)
@@ -349,7 +349,8 @@ unsigned long vm_mmap(struct file *file, unsigned long addr,
if (unlikely(offset_in_page(offset)))
return -EINVAL;
- return vm_mmap_pgoff(file, addr, len, prot, flag, offset >> PAGE_SHIFT);
+ return vm_mmap_pgoff(file, addr, len, prot, flag,
+ offset >> PAGE_SHIFT, -1);
}
EXPORT_SYMBOL(vm_mmap);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 03/14] fs: MAP_DIRECT core
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:49 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: J. Bruce Fields, Jan Kara, Darrick J. Wong, linux-rdma,
linux-api, Dave Chinner, linux-xfs, linux-mm, iommu,
linux-fsdevel, Jeff Layton, Christoph Hellwig
Introduce a set of helper apis for filesystems to establish FL_LAYOUT
leases to protect against writes and block map updates while a
MAP_DIRECT mapping is established. While the lease protects against the
syscall write path and fallocate it does not protect against allocating
write-faults, so this relies on i_mapdcount to disable block map updates
from write faults.
Like the pnfs case MAP_DIRECT does its own timeout of the lease since we
need to have a process context for running map_direct_invalidate().
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/Kconfig | 1
fs/Makefile | 2
fs/mapdirect.c | 237 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 40 ++++++++
4 files changed, 279 insertions(+), 1 deletion(-)
create mode 100644 fs/mapdirect.c
create mode 100644 include/linux/mapdirect.h
diff --git a/fs/Kconfig b/fs/Kconfig
index 7aee6d699fd6..a7b31a96a753 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -37,6 +37,7 @@ source "fs/f2fs/Kconfig"
config FS_DAX
bool "Direct Access (DAX) support"
depends on MMU
+ depends on FILE_LOCKING
depends on !(ARM || MIPS || SPARC)
select FS_IOMAP
select DAX
diff --git a/fs/Makefile b/fs/Makefile
index 7bbaca9c67b1..c0e791d235d8 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -29,7 +29,7 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD) += eventfd.o
obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
obj-$(CONFIG_AIO) += aio.o
-obj-$(CONFIG_FS_DAX) += dax.o
+obj-$(CONFIG_FS_DAX) += dax.o mapdirect.o
obj-$(CONFIG_FS_ENCRYPTION) += crypto/
obj-$(CONFIG_FILE_LOCKING) += locks.o
obj-$(CONFIG_COMPAT) += compat.o compat_ioctl.o
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
new file mode 100644
index 000000000000..9f4dd7395dcd
--- /dev/null
+++ b/fs/mapdirect.c
@@ -0,0 +1,237 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/mapdirect.h>
+#include <linux/workqueue.h>
+#include <linux/signal.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+
+#define MAPDIRECT_BREAK 0
+#define MAPDIRECT_VALID 1
+
+struct map_direct_state {
+ atomic_t mds_ref;
+ atomic_t mds_vmaref;
+ unsigned long mds_state;
+ struct inode *mds_inode;
+ struct delayed_work mds_work;
+ struct fasync_struct *mds_fa;
+ struct vm_area_struct *mds_vma;
+};
+
+bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_VALID, &mds->mds_state);
+}
+EXPORT_SYMBOL_GPL(test_map_direct_valid);
+
+static void put_map_direct(struct map_direct_state *mds)
+{
+ if (!atomic_dec_and_test(&mds->mds_ref))
+ return;
+ kfree(mds);
+}
+
+static void put_map_direct_vma(struct map_direct_state *mds)
+{
+ struct vm_area_struct *vma = mds->mds_vma;
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ void *owner = mds;
+
+ if (!atomic_dec_and_test(&mds->mds_vmaref))
+ return;
+
+ /*
+ * Flush in-flight+forced lm_break events that may be
+ * referencing this dying vma.
+ */
+ mds->mds_vma = NULL;
+ set_bit(MAPDIRECT_BREAK, &mds->mds_state);
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&mds->mds_work);
+ iput(inode);
+
+ put_map_direct(mds);
+}
+
+void generic_map_direct_close(struct vm_area_struct *vma)
+{
+ put_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_close);
+
+static void get_map_direct_vma(struct map_direct_state *mds)
+{
+ atomic_inc(&mds->mds_vmaref);
+}
+
+void generic_map_direct_open(struct vm_area_struct *vma)
+{
+ get_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_open);
+
+static void map_direct_invalidate(struct work_struct *work)
+{
+ struct map_direct_state *mds;
+ struct vm_area_struct *vma;
+ struct inode *inode;
+ void *owner;
+
+ mds = container_of(work, typeof(*mds), mds_work.work);
+
+ clear_bit(MAPDIRECT_VALID, &mds->mds_state);
+
+ vma = ACCESS_ONCE(mds->mds_vma);
+ inode = mds->mds_inode;
+ if (vma) {
+ unsigned long len = vma->vm_end - vma->vm_start;
+ loff_t start = (loff_t) vma->vm_pgoff * PAGE_SIZE;
+
+ unmap_mapping_range(inode->i_mapping, start, len, 1);
+ }
+ owner = mds;
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+
+ put_map_direct(mds);
+}
+
+static bool map_direct_lm_break(struct file_lock *fl)
+{
+ struct map_direct_state *mds = fl->fl_owner;
+
+ /*
+ * Given that we need to take sleeping locks to invalidate the
+ * mapping we schedule that work with the original timeout set
+ * by the file-locks core. Then we tell the core to hold off on
+ * continuing with the lease break until the delayed work
+ * completes the invalidation and the lease unlock.
+ *
+ * Note that this assumes that i_mapdcount is protecting against
+ * block-map modifying write-faults since we are unable to use
+ * leases in that path due to locking constraints.
+ */
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &mds->mds_state)) {
+ schedule_delayed_work(&mds->mds_work, lease_break_time * HZ);
+ kill_fasync(&fl->fl_fasync, SIGIO, POLL_MSG);
+ }
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int map_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+
+ return lease_modify(fl, arg, dispose);
+}
+
+static void map_direct_lm_setup(struct file_lock *fl, void **priv)
+{
+ struct file *file = fl->fl_file;
+ struct map_direct_state *mds = *priv;
+ struct fasync_struct *fa = mds->mds_fa;
+
+ /*
+ * Comment copied from lease_setup():
+ * fasync_insert_entry() returns the old entry if any. If there was no
+ * old entry, then it used "priv" and inserted it into the fasync list.
+ * Clear the pointer to indicate that it shouldn't be freed.
+ */
+ if (!fasync_insert_entry(fa->fa_fd, file, &fl->fl_fasync, fa))
+ *priv = NULL;
+
+ __f_setown(file, task_pid(current), PIDTYPE_PID, 0);
+}
+
+static const struct lock_manager_operations map_direct_lm_ops = {
+ .lm_break = map_direct_lm_break,
+ .lm_change = map_direct_lm_change,
+ .lm_setup = map_direct_lm_setup,
+};
+
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
+{
+ struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ struct fasync_struct *fa;
+ struct file_lock *fl;
+ void *owner = mds;
+ int rc = -ENOMEM;
+
+ if (!mds)
+ return ERR_PTR(-ENOMEM);
+
+ mds->mds_vma = vma;
+ atomic_set(&mds->mds_ref, 1);
+ atomic_set(&mds->mds_vmaref, 1);
+ set_bit(MAPDIRECT_VALID, &mds->mds_state);
+ mds->mds_inode = inode;
+ ihold(inode);
+ INIT_DELAYED_WORK(&mds->mds_work, map_direct_invalidate);
+
+ fa = fasync_alloc();
+ if (!fa)
+ goto err_fasync_alloc;
+ mds->mds_fa = fa;
+ fa->fa_fd = fd;
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &map_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = mds;
+ atomic_inc(&mds->mds_ref);
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = mds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return mds;
+
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ /* if owner is NULL then the lease machinery is reponsible @fa */
+ if (owner)
+ fasync_free(fa);
+err_fasync_alloc:
+ iput(inode);
+ kfree(mds);
+ return ERR_PTR(rc);
+}
+EXPORT_SYMBOL_GPL(map_direct_register);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
new file mode 100644
index 000000000000..5491aa550e55
--- /dev/null
+++ b/include/linux/mapdirect.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#ifndef __MAPDIRECT_H__
+#define __MAPDIRECT_H__
+#include <linux/err.h>
+
+struct inode;
+struct work_struct;
+struct vm_area_struct;
+struct map_direct_state;
+
+#if IS_ENABLED(CONFIG_FS_DAX)
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
+bool test_map_direct_valid(struct map_direct_state *mds);
+void generic_map_direct_open(struct vm_area_struct *vma);
+void generic_map_direct_close(struct vm_area_struct *vma);
+#else
+static inline struct map_direct_state *map_direct_register(int fd,
+ struct vm_area_struct *vma)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+static inline bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return false;
+}
+#define generic_map_direct_open NULL
+#define generic_map_direct_close NULL
+#endif
+#endif /* __MAPDIRECT_H__ */
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 03/14] fs: MAP_DIRECT core
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
Introduce a set of helper apis for filesystems to establish FL_LAYOUT
leases to protect against writes and block map updates while a
MAP_DIRECT mapping is established. While the lease protects against the
syscall write path and fallocate it does not protect against allocating
write-faults, so this relies on i_mapdcount to disable block map updates
from write faults.
Like the pnfs case MAP_DIRECT does its own timeout of the lease since we
need to have a process context for running map_direct_invalidate().
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/Kconfig | 1
fs/Makefile | 2
fs/mapdirect.c | 237 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 40 ++++++++
4 files changed, 279 insertions(+), 1 deletion(-)
create mode 100644 fs/mapdirect.c
create mode 100644 include/linux/mapdirect.h
diff --git a/fs/Kconfig b/fs/Kconfig
index 7aee6d699fd6..a7b31a96a753 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -37,6 +37,7 @@ source "fs/f2fs/Kconfig"
config FS_DAX
bool "Direct Access (DAX) support"
depends on MMU
+ depends on FILE_LOCKING
depends on !(ARM || MIPS || SPARC)
select FS_IOMAP
select DAX
diff --git a/fs/Makefile b/fs/Makefile
index 7bbaca9c67b1..c0e791d235d8 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -29,7 +29,7 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD) += eventfd.o
obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
obj-$(CONFIG_AIO) += aio.o
-obj-$(CONFIG_FS_DAX) += dax.o
+obj-$(CONFIG_FS_DAX) += dax.o mapdirect.o
obj-$(CONFIG_FS_ENCRYPTION) += crypto/
obj-$(CONFIG_FILE_LOCKING) += locks.o
obj-$(CONFIG_COMPAT) += compat.o compat_ioctl.o
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
new file mode 100644
index 000000000000..9f4dd7395dcd
--- /dev/null
+++ b/fs/mapdirect.c
@@ -0,0 +1,237 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/mapdirect.h>
+#include <linux/workqueue.h>
+#include <linux/signal.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+
+#define MAPDIRECT_BREAK 0
+#define MAPDIRECT_VALID 1
+
+struct map_direct_state {
+ atomic_t mds_ref;
+ atomic_t mds_vmaref;
+ unsigned long mds_state;
+ struct inode *mds_inode;
+ struct delayed_work mds_work;
+ struct fasync_struct *mds_fa;
+ struct vm_area_struct *mds_vma;
+};
+
+bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_VALID, &mds->mds_state);
+}
+EXPORT_SYMBOL_GPL(test_map_direct_valid);
+
+static void put_map_direct(struct map_direct_state *mds)
+{
+ if (!atomic_dec_and_test(&mds->mds_ref))
+ return;
+ kfree(mds);
+}
+
+static void put_map_direct_vma(struct map_direct_state *mds)
+{
+ struct vm_area_struct *vma = mds->mds_vma;
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ void *owner = mds;
+
+ if (!atomic_dec_and_test(&mds->mds_vmaref))
+ return;
+
+ /*
+ * Flush in-flight+forced lm_break events that may be
+ * referencing this dying vma.
+ */
+ mds->mds_vma = NULL;
+ set_bit(MAPDIRECT_BREAK, &mds->mds_state);
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&mds->mds_work);
+ iput(inode);
+
+ put_map_direct(mds);
+}
+
+void generic_map_direct_close(struct vm_area_struct *vma)
+{
+ put_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_close);
+
+static void get_map_direct_vma(struct map_direct_state *mds)
+{
+ atomic_inc(&mds->mds_vmaref);
+}
+
+void generic_map_direct_open(struct vm_area_struct *vma)
+{
+ get_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_open);
+
+static void map_direct_invalidate(struct work_struct *work)
+{
+ struct map_direct_state *mds;
+ struct vm_area_struct *vma;
+ struct inode *inode;
+ void *owner;
+
+ mds = container_of(work, typeof(*mds), mds_work.work);
+
+ clear_bit(MAPDIRECT_VALID, &mds->mds_state);
+
+ vma = ACCESS_ONCE(mds->mds_vma);
+ inode = mds->mds_inode;
+ if (vma) {
+ unsigned long len = vma->vm_end - vma->vm_start;
+ loff_t start = (loff_t) vma->vm_pgoff * PAGE_SIZE;
+
+ unmap_mapping_range(inode->i_mapping, start, len, 1);
+ }
+ owner = mds;
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+
+ put_map_direct(mds);
+}
+
+static bool map_direct_lm_break(struct file_lock *fl)
+{
+ struct map_direct_state *mds = fl->fl_owner;
+
+ /*
+ * Given that we need to take sleeping locks to invalidate the
+ * mapping we schedule that work with the original timeout set
+ * by the file-locks core. Then we tell the core to hold off on
+ * continuing with the lease break until the delayed work
+ * completes the invalidation and the lease unlock.
+ *
+ * Note that this assumes that i_mapdcount is protecting against
+ * block-map modifying write-faults since we are unable to use
+ * leases in that path due to locking constraints.
+ */
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &mds->mds_state)) {
+ schedule_delayed_work(&mds->mds_work, lease_break_time * HZ);
+ kill_fasync(&fl->fl_fasync, SIGIO, POLL_MSG);
+ }
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int map_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+
+ return lease_modify(fl, arg, dispose);
+}
+
+static void map_direct_lm_setup(struct file_lock *fl, void **priv)
+{
+ struct file *file = fl->fl_file;
+ struct map_direct_state *mds = *priv;
+ struct fasync_struct *fa = mds->mds_fa;
+
+ /*
+ * Comment copied from lease_setup():
+ * fasync_insert_entry() returns the old entry if any. If there was no
+ * old entry, then it used "priv" and inserted it into the fasync list.
+ * Clear the pointer to indicate that it shouldn't be freed.
+ */
+ if (!fasync_insert_entry(fa->fa_fd, file, &fl->fl_fasync, fa))
+ *priv = NULL;
+
+ __f_setown(file, task_pid(current), PIDTYPE_PID, 0);
+}
+
+static const struct lock_manager_operations map_direct_lm_ops = {
+ .lm_break = map_direct_lm_break,
+ .lm_change = map_direct_lm_change,
+ .lm_setup = map_direct_lm_setup,
+};
+
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
+{
+ struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ struct fasync_struct *fa;
+ struct file_lock *fl;
+ void *owner = mds;
+ int rc = -ENOMEM;
+
+ if (!mds)
+ return ERR_PTR(-ENOMEM);
+
+ mds->mds_vma = vma;
+ atomic_set(&mds->mds_ref, 1);
+ atomic_set(&mds->mds_vmaref, 1);
+ set_bit(MAPDIRECT_VALID, &mds->mds_state);
+ mds->mds_inode = inode;
+ ihold(inode);
+ INIT_DELAYED_WORK(&mds->mds_work, map_direct_invalidate);
+
+ fa = fasync_alloc();
+ if (!fa)
+ goto err_fasync_alloc;
+ mds->mds_fa = fa;
+ fa->fa_fd = fd;
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &map_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = mds;
+ atomic_inc(&mds->mds_ref);
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = mds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return mds;
+
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ /* if owner is NULL then the lease machinery is reponsible @fa */
+ if (owner)
+ fasync_free(fa);
+err_fasync_alloc:
+ iput(inode);
+ kfree(mds);
+ return ERR_PTR(rc);
+}
+EXPORT_SYMBOL_GPL(map_direct_register);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
new file mode 100644
index 000000000000..5491aa550e55
--- /dev/null
+++ b/include/linux/mapdirect.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#ifndef __MAPDIRECT_H__
+#define __MAPDIRECT_H__
+#include <linux/err.h>
+
+struct inode;
+struct work_struct;
+struct vm_area_struct;
+struct map_direct_state;
+
+#if IS_ENABLED(CONFIG_FS_DAX)
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
+bool test_map_direct_valid(struct map_direct_state *mds);
+void generic_map_direct_open(struct vm_area_struct *vma);
+void generic_map_direct_close(struct vm_area_struct *vma);
+#else
+static inline struct map_direct_state *map_direct_register(int fd,
+ struct vm_area_struct *vma)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+static inline bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return false;
+}
+#define generic_map_direct_open NULL
+#define generic_map_direct_close NULL
+#endif
+#endif /* __MAPDIRECT_H__ */
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 03/14] fs: MAP_DIRECT core
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
Introduce a set of helper apis for filesystems to establish FL_LAYOUT
leases to protect against writes and block map updates while a
MAP_DIRECT mapping is established. While the lease protects against the
syscall write path and fallocate it does not protect against allocating
write-faults, so this relies on i_mapdcount to disable block map updates
from write faults.
Like the pnfs case MAP_DIRECT does its own timeout of the lease since we
need to have a process context for running map_direct_invalidate().
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/Kconfig | 1
fs/Makefile | 2
fs/mapdirect.c | 237 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 40 ++++++++
4 files changed, 279 insertions(+), 1 deletion(-)
create mode 100644 fs/mapdirect.c
create mode 100644 include/linux/mapdirect.h
diff --git a/fs/Kconfig b/fs/Kconfig
index 7aee6d699fd6..a7b31a96a753 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -37,6 +37,7 @@ source "fs/f2fs/Kconfig"
config FS_DAX
bool "Direct Access (DAX) support"
depends on MMU
+ depends on FILE_LOCKING
depends on !(ARM || MIPS || SPARC)
select FS_IOMAP
select DAX
diff --git a/fs/Makefile b/fs/Makefile
index 7bbaca9c67b1..c0e791d235d8 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -29,7 +29,7 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD) += eventfd.o
obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
obj-$(CONFIG_AIO) += aio.o
-obj-$(CONFIG_FS_DAX) += dax.o
+obj-$(CONFIG_FS_DAX) += dax.o mapdirect.o
obj-$(CONFIG_FS_ENCRYPTION) += crypto/
obj-$(CONFIG_FILE_LOCKING) += locks.o
obj-$(CONFIG_COMPAT) += compat.o compat_ioctl.o
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
new file mode 100644
index 000000000000..9f4dd7395dcd
--- /dev/null
+++ b/fs/mapdirect.c
@@ -0,0 +1,237 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/mapdirect.h>
+#include <linux/workqueue.h>
+#include <linux/signal.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+
+#define MAPDIRECT_BREAK 0
+#define MAPDIRECT_VALID 1
+
+struct map_direct_state {
+ atomic_t mds_ref;
+ atomic_t mds_vmaref;
+ unsigned long mds_state;
+ struct inode *mds_inode;
+ struct delayed_work mds_work;
+ struct fasync_struct *mds_fa;
+ struct vm_area_struct *mds_vma;
+};
+
+bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_VALID, &mds->mds_state);
+}
+EXPORT_SYMBOL_GPL(test_map_direct_valid);
+
+static void put_map_direct(struct map_direct_state *mds)
+{
+ if (!atomic_dec_and_test(&mds->mds_ref))
+ return;
+ kfree(mds);
+}
+
+static void put_map_direct_vma(struct map_direct_state *mds)
+{
+ struct vm_area_struct *vma = mds->mds_vma;
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ void *owner = mds;
+
+ if (!atomic_dec_and_test(&mds->mds_vmaref))
+ return;
+
+ /*
+ * Flush in-flight+forced lm_break events that may be
+ * referencing this dying vma.
+ */
+ mds->mds_vma = NULL;
+ set_bit(MAPDIRECT_BREAK, &mds->mds_state);
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&mds->mds_work);
+ iput(inode);
+
+ put_map_direct(mds);
+}
+
+void generic_map_direct_close(struct vm_area_struct *vma)
+{
+ put_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_close);
+
+static void get_map_direct_vma(struct map_direct_state *mds)
+{
+ atomic_inc(&mds->mds_vmaref);
+}
+
+void generic_map_direct_open(struct vm_area_struct *vma)
+{
+ get_map_direct_vma(vma->vm_private_data);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_open);
+
+static void map_direct_invalidate(struct work_struct *work)
+{
+ struct map_direct_state *mds;
+ struct vm_area_struct *vma;
+ struct inode *inode;
+ void *owner;
+
+ mds = container_of(work, typeof(*mds), mds_work.work);
+
+ clear_bit(MAPDIRECT_VALID, &mds->mds_state);
+
+ vma = ACCESS_ONCE(mds->mds_vma);
+ inode = mds->mds_inode;
+ if (vma) {
+ unsigned long len = vma->vm_end - vma->vm_start;
+ loff_t start = (loff_t) vma->vm_pgoff * PAGE_SIZE;
+
+ unmap_mapping_range(inode->i_mapping, start, len, 1);
+ }
+ owner = mds;
+ vfs_setlease(vma->vm_file, F_UNLCK, NULL, &owner);
+
+ put_map_direct(mds);
+}
+
+static bool map_direct_lm_break(struct file_lock *fl)
+{
+ struct map_direct_state *mds = fl->fl_owner;
+
+ /*
+ * Given that we need to take sleeping locks to invalidate the
+ * mapping we schedule that work with the original timeout set
+ * by the file-locks core. Then we tell the core to hold off on
+ * continuing with the lease break until the delayed work
+ * completes the invalidation and the lease unlock.
+ *
+ * Note that this assumes that i_mapdcount is protecting against
+ * block-map modifying write-faults since we are unable to use
+ * leases in that path due to locking constraints.
+ */
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &mds->mds_state)) {
+ schedule_delayed_work(&mds->mds_work, lease_break_time * HZ);
+ kill_fasync(&fl->fl_fasync, SIGIO, POLL_MSG);
+ }
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int map_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+
+ return lease_modify(fl, arg, dispose);
+}
+
+static void map_direct_lm_setup(struct file_lock *fl, void **priv)
+{
+ struct file *file = fl->fl_file;
+ struct map_direct_state *mds = *priv;
+ struct fasync_struct *fa = mds->mds_fa;
+
+ /*
+ * Comment copied from lease_setup():
+ * fasync_insert_entry() returns the old entry if any. If there was no
+ * old entry, then it used "priv" and inserted it into the fasync list.
+ * Clear the pointer to indicate that it shouldn't be freed.
+ */
+ if (!fasync_insert_entry(fa->fa_fd, file, &fl->fl_fasync, fa))
+ *priv = NULL;
+
+ __f_setown(file, task_pid(current), PIDTYPE_PID, 0);
+}
+
+static const struct lock_manager_operations map_direct_lm_ops = {
+ .lm_break = map_direct_lm_break,
+ .lm_change = map_direct_lm_change,
+ .lm_setup = map_direct_lm_setup,
+};
+
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
+{
+ struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
+ struct file *file = vma->vm_file;
+ struct inode *inode = file_inode(file);
+ struct fasync_struct *fa;
+ struct file_lock *fl;
+ void *owner = mds;
+ int rc = -ENOMEM;
+
+ if (!mds)
+ return ERR_PTR(-ENOMEM);
+
+ mds->mds_vma = vma;
+ atomic_set(&mds->mds_ref, 1);
+ atomic_set(&mds->mds_vmaref, 1);
+ set_bit(MAPDIRECT_VALID, &mds->mds_state);
+ mds->mds_inode = inode;
+ ihold(inode);
+ INIT_DELAYED_WORK(&mds->mds_work, map_direct_invalidate);
+
+ fa = fasync_alloc();
+ if (!fa)
+ goto err_fasync_alloc;
+ mds->mds_fa = fa;
+ fa->fa_fd = fd;
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &map_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = mds;
+ atomic_inc(&mds->mds_ref);
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = mds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return mds;
+
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ /* if owner is NULL then the lease machinery is reponsible @fa */
+ if (owner)
+ fasync_free(fa);
+err_fasync_alloc:
+ iput(inode);
+ kfree(mds);
+ return ERR_PTR(rc);
+}
+EXPORT_SYMBOL_GPL(map_direct_register);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
new file mode 100644
index 000000000000..5491aa550e55
--- /dev/null
+++ b/include/linux/mapdirect.h
@@ -0,0 +1,40 @@
+/*
+ * Copyright(c) 2017 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#ifndef __MAPDIRECT_H__
+#define __MAPDIRECT_H__
+#include <linux/err.h>
+
+struct inode;
+struct work_struct;
+struct vm_area_struct;
+struct map_direct_state;
+
+#if IS_ENABLED(CONFIG_FS_DAX)
+struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
+bool test_map_direct_valid(struct map_direct_state *mds);
+void generic_map_direct_open(struct vm_area_struct *vma);
+void generic_map_direct_close(struct vm_area_struct *vma);
+#else
+static inline struct map_direct_state *map_direct_register(int fd,
+ struct vm_area_struct *vma)
+{
+ return ERR_PTR(-EOPNOTSUPP);
+}
+static inline bool test_map_direct_valid(struct map_direct_state *mds)
+{
+ return false;
+}
+#define generic_map_direct_open NULL
+#define generic_map_direct_close NULL
+#endif
+#endif /* __MAPDIRECT_H__ */
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:49 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
linux-xfs, linux-mm, iommu, linux-fsdevel, Christoph Hellwig
Move xfs_break_layouts() to its own compilation unit so that it can be
used for both pnfs layouts and MAP_DIRECT mappings.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 4 ++++
fs/xfs/Makefile | 1 +
fs/xfs/xfs_layout.c | 42 ++++++++++++++++++++++++++++++++++++++++++
fs/xfs/xfs_layout.h | 13 +++++++++++++
fs/xfs/xfs_pnfs.c | 30 ------------------------------
fs/xfs/xfs_pnfs.h | 10 ++--------
6 files changed, 62 insertions(+), 38 deletions(-)
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index 1b98cfa342ab..f62fc6629abb 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -109,3 +109,7 @@ config XFS_ASSERT_FATAL
result in warnings.
This behavior can be modified at runtime via sysfs.
+
+config XFS_LAYOUT
+ def_bool y
+ depends on EXPORTFS_BLOCK_OPS
diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile
index a6e955bfead8..d44135107490 100644
--- a/fs/xfs/Makefile
+++ b/fs/xfs/Makefile
@@ -135,3 +135,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o
xfs-$(CONFIG_SYSCTL) += xfs_sysctl.o
xfs-$(CONFIG_COMPAT) += xfs_ioctl32.o
xfs-$(CONFIG_EXPORTFS_BLOCK_OPS) += xfs_pnfs.o
+xfs-$(CONFIG_XFS_LAYOUT) += xfs_layout.o
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
new file mode 100644
index 000000000000..71d95e1a910a
--- /dev/null
+++ b/fs/xfs/xfs_layout.c
@@ -0,0 +1,42 @@
+/*
+ * Copyright (c) 2014 Christoph Hellwig.
+ */
+#include "xfs.h"
+#include "xfs_format.h"
+#include "xfs_log_format.h"
+#include "xfs_trans_resv.h"
+#include "xfs_sb.h"
+#include "xfs_mount.h"
+#include "xfs_inode.h"
+
+#include <linux/fs.h>
+
+/*
+ * Ensure that we do not have any outstanding pNFS layouts that can be used by
+ * clients to directly read from or write to this inode. This must be called
+ * before every operation that can remove blocks from the extent map.
+ * Additionally we call it during the write operation, where aren't concerned
+ * about exposing unallocated blocks but just want to provide basic
+ * synchronization between a local writer and pNFS clients. mmap writes would
+ * also benefit from this sort of synchronization, but due to the tricky locking
+ * rules in the page fault path we don't bother.
+ */
+int
+xfs_break_layouts(
+ struct inode *inode,
+ uint *iolock)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ int error;
+
+ ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
+
+ while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
+ xfs_iunlock(ip, *iolock);
+ error = break_layout(inode, true);
+ *iolock = XFS_IOLOCK_EXCL;
+ xfs_ilock(ip, *iolock);
+ }
+
+ return error;
+}
diff --git a/fs/xfs/xfs_layout.h b/fs/xfs/xfs_layout.h
new file mode 100644
index 000000000000..f848ee78cc93
--- /dev/null
+++ b/fs/xfs/xfs_layout.h
@@ -0,0 +1,13 @@
+#ifndef _XFS_LAYOUT_H
+#define _XFS_LAYOUT_H 1
+
+#ifdef CONFIG_XFS_LAYOUT
+int xfs_break_layouts(struct inode *inode, uint *iolock);
+#else
+static inline int
+xfs_break_layouts(struct inode *inode, uint *iolock)
+{
+ return 0;
+}
+#endif /* CONFIG_XFS_LAYOUT */
+#endif /* _XFS_LAYOUT_H */
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index 2f2dc3c09ad0..8ec72220e73b 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -20,36 +20,6 @@
#include "xfs_pnfs.h"
/*
- * Ensure that we do not have any outstanding pNFS layouts that can be used by
- * clients to directly read from or write to this inode. This must be called
- * before every operation that can remove blocks from the extent map.
- * Additionally we call it during the write operation, where aren't concerned
- * about exposing unallocated blocks but just want to provide basic
- * synchronization between a local writer and pNFS clients. mmap writes would
- * also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
- */
-int
-xfs_break_layouts(
- struct inode *inode,
- uint *iolock)
-{
- struct xfs_inode *ip = XFS_I(inode);
- int error;
-
- ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
-
- while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
- xfs_iunlock(ip, *iolock);
- error = break_layout(inode, true);
- *iolock = XFS_IOLOCK_EXCL;
- xfs_ilock(ip, *iolock);
- }
-
- return error;
-}
-
-/*
* Get a unique ID including its location so that the client can identify
* the exported device.
*/
diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
index b587cb99b2b7..4135b2482697 100644
--- a/fs/xfs/xfs_pnfs.h
+++ b/fs/xfs/xfs_pnfs.h
@@ -1,19 +1,13 @@
#ifndef _XFS_PNFS_H
#define _XFS_PNFS_H 1
+#include "xfs_layout.h"
+
#ifdef CONFIG_EXPORTFS_BLOCK_OPS
int xfs_fs_get_uuid(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
struct iomap *iomap, bool write, u32 *device_generation);
int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
struct iattr *iattr);
-
-int xfs_break_layouts(struct inode *inode, uint *iolock);
-#else
-static inline int
-xfs_break_layouts(struct inode *inode, uint *iolock)
-{
- return 0;
-}
#endif /* CONFIG_EXPORTFS_BLOCK_OPS */
#endif /* _XFS_PNFS_H */
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
Move xfs_break_layouts() to its own compilation unit so that it can be
used for both pnfs layouts and MAP_DIRECT mappings.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 4 ++++
fs/xfs/Makefile | 1 +
fs/xfs/xfs_layout.c | 42 ++++++++++++++++++++++++++++++++++++++++++
fs/xfs/xfs_layout.h | 13 +++++++++++++
fs/xfs/xfs_pnfs.c | 30 ------------------------------
fs/xfs/xfs_pnfs.h | 10 ++--------
6 files changed, 62 insertions(+), 38 deletions(-)
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index 1b98cfa342ab..f62fc6629abb 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -109,3 +109,7 @@ config XFS_ASSERT_FATAL
result in warnings.
This behavior can be modified at runtime via sysfs.
+
+config XFS_LAYOUT
+ def_bool y
+ depends on EXPORTFS_BLOCK_OPS
diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile
index a6e955bfead8..d44135107490 100644
--- a/fs/xfs/Makefile
+++ b/fs/xfs/Makefile
@@ -135,3 +135,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o
xfs-$(CONFIG_SYSCTL) += xfs_sysctl.o
xfs-$(CONFIG_COMPAT) += xfs_ioctl32.o
xfs-$(CONFIG_EXPORTFS_BLOCK_OPS) += xfs_pnfs.o
+xfs-$(CONFIG_XFS_LAYOUT) += xfs_layout.o
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
new file mode 100644
index 000000000000..71d95e1a910a
--- /dev/null
+++ b/fs/xfs/xfs_layout.c
@@ -0,0 +1,42 @@
+/*
+ * Copyright (c) 2014 Christoph Hellwig.
+ */
+#include "xfs.h"
+#include "xfs_format.h"
+#include "xfs_log_format.h"
+#include "xfs_trans_resv.h"
+#include "xfs_sb.h"
+#include "xfs_mount.h"
+#include "xfs_inode.h"
+
+#include <linux/fs.h>
+
+/*
+ * Ensure that we do not have any outstanding pNFS layouts that can be used by
+ * clients to directly read from or write to this inode. This must be called
+ * before every operation that can remove blocks from the extent map.
+ * Additionally we call it during the write operation, where aren't concerned
+ * about exposing unallocated blocks but just want to provide basic
+ * synchronization between a local writer and pNFS clients. mmap writes would
+ * also benefit from this sort of synchronization, but due to the tricky locking
+ * rules in the page fault path we don't bother.
+ */
+int
+xfs_break_layouts(
+ struct inode *inode,
+ uint *iolock)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ int error;
+
+ ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
+
+ while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
+ xfs_iunlock(ip, *iolock);
+ error = break_layout(inode, true);
+ *iolock = XFS_IOLOCK_EXCL;
+ xfs_ilock(ip, *iolock);
+ }
+
+ return error;
+}
diff --git a/fs/xfs/xfs_layout.h b/fs/xfs/xfs_layout.h
new file mode 100644
index 000000000000..f848ee78cc93
--- /dev/null
+++ b/fs/xfs/xfs_layout.h
@@ -0,0 +1,13 @@
+#ifndef _XFS_LAYOUT_H
+#define _XFS_LAYOUT_H 1
+
+#ifdef CONFIG_XFS_LAYOUT
+int xfs_break_layouts(struct inode *inode, uint *iolock);
+#else
+static inline int
+xfs_break_layouts(struct inode *inode, uint *iolock)
+{
+ return 0;
+}
+#endif /* CONFIG_XFS_LAYOUT */
+#endif /* _XFS_LAYOUT_H */
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index 2f2dc3c09ad0..8ec72220e73b 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -20,36 +20,6 @@
#include "xfs_pnfs.h"
/*
- * Ensure that we do not have any outstanding pNFS layouts that can be used by
- * clients to directly read from or write to this inode. This must be called
- * before every operation that can remove blocks from the extent map.
- * Additionally we call it during the write operation, where aren't concerned
- * about exposing unallocated blocks but just want to provide basic
- * synchronization between a local writer and pNFS clients. mmap writes would
- * also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
- */
-int
-xfs_break_layouts(
- struct inode *inode,
- uint *iolock)
-{
- struct xfs_inode *ip = XFS_I(inode);
- int error;
-
- ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
-
- while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
- xfs_iunlock(ip, *iolock);
- error = break_layout(inode, true);
- *iolock = XFS_IOLOCK_EXCL;
- xfs_ilock(ip, *iolock);
- }
-
- return error;
-}
-
-/*
* Get a unique ID including its location so that the client can identify
* the exported device.
*/
diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
index b587cb99b2b7..4135b2482697 100644
--- a/fs/xfs/xfs_pnfs.h
+++ b/fs/xfs/xfs_pnfs.h
@@ -1,19 +1,13 @@
#ifndef _XFS_PNFS_H
#define _XFS_PNFS_H 1
+#include "xfs_layout.h"
+
#ifdef CONFIG_EXPORTFS_BLOCK_OPS
int xfs_fs_get_uuid(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
struct iomap *iomap, bool write, u32 *device_generation);
int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
struct iattr *iattr);
-
-int xfs_break_layouts(struct inode *inode, uint *iolock);
-#else
-static inline int
-xfs_break_layouts(struct inode *inode, uint *iolock)
-{
- return 0;
-}
#endif /* CONFIG_EXPORTFS_BLOCK_OPS */
#endif /* _XFS_PNFS_H */
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
Move xfs_break_layouts() to its own compilation unit so that it can be
used for both pnfs layouts and MAP_DIRECT mappings.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 4 ++++
fs/xfs/Makefile | 1 +
fs/xfs/xfs_layout.c | 42 ++++++++++++++++++++++++++++++++++++++++++
fs/xfs/xfs_layout.h | 13 +++++++++++++
fs/xfs/xfs_pnfs.c | 30 ------------------------------
fs/xfs/xfs_pnfs.h | 10 ++--------
6 files changed, 62 insertions(+), 38 deletions(-)
create mode 100644 fs/xfs/xfs_layout.c
create mode 100644 fs/xfs/xfs_layout.h
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index 1b98cfa342ab..f62fc6629abb 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -109,3 +109,7 @@ config XFS_ASSERT_FATAL
result in warnings.
This behavior can be modified at runtime via sysfs.
+
+config XFS_LAYOUT
+ def_bool y
+ depends on EXPORTFS_BLOCK_OPS
diff --git a/fs/xfs/Makefile b/fs/xfs/Makefile
index a6e955bfead8..d44135107490 100644
--- a/fs/xfs/Makefile
+++ b/fs/xfs/Makefile
@@ -135,3 +135,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o
xfs-$(CONFIG_SYSCTL) += xfs_sysctl.o
xfs-$(CONFIG_COMPAT) += xfs_ioctl32.o
xfs-$(CONFIG_EXPORTFS_BLOCK_OPS) += xfs_pnfs.o
+xfs-$(CONFIG_XFS_LAYOUT) += xfs_layout.o
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
new file mode 100644
index 000000000000..71d95e1a910a
--- /dev/null
+++ b/fs/xfs/xfs_layout.c
@@ -0,0 +1,42 @@
+/*
+ * Copyright (c) 2014 Christoph Hellwig.
+ */
+#include "xfs.h"
+#include "xfs_format.h"
+#include "xfs_log_format.h"
+#include "xfs_trans_resv.h"
+#include "xfs_sb.h"
+#include "xfs_mount.h"
+#include "xfs_inode.h"
+
+#include <linux/fs.h>
+
+/*
+ * Ensure that we do not have any outstanding pNFS layouts that can be used by
+ * clients to directly read from or write to this inode. This must be called
+ * before every operation that can remove blocks from the extent map.
+ * Additionally we call it during the write operation, where aren't concerned
+ * about exposing unallocated blocks but just want to provide basic
+ * synchronization between a local writer and pNFS clients. mmap writes would
+ * also benefit from this sort of synchronization, but due to the tricky locking
+ * rules in the page fault path we don't bother.
+ */
+int
+xfs_break_layouts(
+ struct inode *inode,
+ uint *iolock)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ int error;
+
+ ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
+
+ while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
+ xfs_iunlock(ip, *iolock);
+ error = break_layout(inode, true);
+ *iolock = XFS_IOLOCK_EXCL;
+ xfs_ilock(ip, *iolock);
+ }
+
+ return error;
+}
diff --git a/fs/xfs/xfs_layout.h b/fs/xfs/xfs_layout.h
new file mode 100644
index 000000000000..f848ee78cc93
--- /dev/null
+++ b/fs/xfs/xfs_layout.h
@@ -0,0 +1,13 @@
+#ifndef _XFS_LAYOUT_H
+#define _XFS_LAYOUT_H 1
+
+#ifdef CONFIG_XFS_LAYOUT
+int xfs_break_layouts(struct inode *inode, uint *iolock);
+#else
+static inline int
+xfs_break_layouts(struct inode *inode, uint *iolock)
+{
+ return 0;
+}
+#endif /* CONFIG_XFS_LAYOUT */
+#endif /* _XFS_LAYOUT_H */
diff --git a/fs/xfs/xfs_pnfs.c b/fs/xfs/xfs_pnfs.c
index 2f2dc3c09ad0..8ec72220e73b 100644
--- a/fs/xfs/xfs_pnfs.c
+++ b/fs/xfs/xfs_pnfs.c
@@ -20,36 +20,6 @@
#include "xfs_pnfs.h"
/*
- * Ensure that we do not have any outstanding pNFS layouts that can be used by
- * clients to directly read from or write to this inode. This must be called
- * before every operation that can remove blocks from the extent map.
- * Additionally we call it during the write operation, where aren't concerned
- * about exposing unallocated blocks but just want to provide basic
- * synchronization between a local writer and pNFS clients. mmap writes would
- * also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
- */
-int
-xfs_break_layouts(
- struct inode *inode,
- uint *iolock)
-{
- struct xfs_inode *ip = XFS_I(inode);
- int error;
-
- ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
-
- while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
- xfs_iunlock(ip, *iolock);
- error = break_layout(inode, true);
- *iolock = XFS_IOLOCK_EXCL;
- xfs_ilock(ip, *iolock);
- }
-
- return error;
-}
-
-/*
* Get a unique ID including its location so that the client can identify
* the exported device.
*/
diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
index b587cb99b2b7..4135b2482697 100644
--- a/fs/xfs/xfs_pnfs.h
+++ b/fs/xfs/xfs_pnfs.h
@@ -1,19 +1,13 @@
#ifndef _XFS_PNFS_H
#define _XFS_PNFS_H 1
+#include "xfs_layout.h"
+
#ifdef CONFIG_EXPORTFS_BLOCK_OPS
int xfs_fs_get_uuid(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
struct iomap *iomap, bool write, u32 *device_generation);
int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
struct iattr *iattr);
-
-int xfs_break_layouts(struct inode *inode, uint *iolock);
-#else
-static inline int
-xfs_break_layouts(struct inode *inode, uint *iolock)
-{
- return 0;
-}
#endif /* CONFIG_EXPORTFS_BLOCK_OPS */
#endif /* _XFS_PNFS_H */
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
2017-10-10 14:49 ` Dan Williams
@ 2017-10-11 0:46 ` Dave Chinner
-1 siblings, 0 replies; 77+ messages in thread
From: Dave Chinner @ 2017-10-11 0:46 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
On Tue, Oct 10, 2017 at 07:49:17AM -0700, Dan Williams wrote:
> Move xfs_break_layouts() to its own compilation unit so that it can be
> used for both pnfs layouts and MAP_DIRECT mappings.
.....
> diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
> index b587cb99b2b7..4135b2482697 100644
> --- a/fs/xfs/xfs_pnfs.h
> +++ b/fs/xfs/xfs_pnfs.h
> @@ -1,19 +1,13 @@
> #ifndef _XFS_PNFS_H
> #define _XFS_PNFS_H 1
>
> +#include "xfs_layout.h"
> +
I missed this the first time through - we try not to put includes
in header files, and instead make sure each C file has all the
includes they require. Can you move this to all the C files that
need layouts and remove the include of the xfs_pnfs.h include from
them?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
@ 2017-10-11 0:46 ` Dave Chinner
0 siblings, 0 replies; 77+ messages in thread
From: Dave Chinner @ 2017-10-11 0:46 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
On Tue, Oct 10, 2017 at 07:49:17AM -0700, Dan Williams wrote:
> Move xfs_break_layouts() to its own compilation unit so that it can be
> used for both pnfs layouts and MAP_DIRECT mappings.
.....
> diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
> index b587cb99b2b7..4135b2482697 100644
> --- a/fs/xfs/xfs_pnfs.h
> +++ b/fs/xfs/xfs_pnfs.h
> @@ -1,19 +1,13 @@
> #ifndef _XFS_PNFS_H
> #define _XFS_PNFS_H 1
>
> +#include "xfs_layout.h"
> +
I missed this the first time through - we try not to put includes
in header files, and instead make sure each C file has all the
includes they require. Can you move this to all the C files that
need layouts and remove the include of the xfs_pnfs.h include from
them?
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
2017-10-11 0:46 ` Dave Chinner
(?)
@ 2017-10-11 2:12 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: Jan Kara, Darrick J. Wong, linux-rdma, Linux API, linux-nvdimm,
linux-xfs, Linux MM, iommu, linux-fsdevel, Christoph Hellwig
On Tue, Oct 10, 2017 at 5:46 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:17AM -0700, Dan Williams wrote:
>> Move xfs_break_layouts() to its own compilation unit so that it can be
>> used for both pnfs layouts and MAP_DIRECT mappings.
> .....
>> diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
>> index b587cb99b2b7..4135b2482697 100644
>> --- a/fs/xfs/xfs_pnfs.h
>> +++ b/fs/xfs/xfs_pnfs.h
>> @@ -1,19 +1,13 @@
>> #ifndef _XFS_PNFS_H
>> #define _XFS_PNFS_H 1
>>
>> +#include "xfs_layout.h"
>> +
>
> I missed this the first time through - we try not to put includes
> in header files, and instead make sure each C file has all the
> includes they require. Can you move this to all the C files that
> need layouts and remove the include of the xfs_pnfs.h include from
> them?
Sure, will do.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
@ 2017-10-11 2:12 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-nvdimm, Jan Kara, Darrick J. Wong, linux-rdma, Linux API,
iommu, linux-xfs, Linux MM, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
On Tue, Oct 10, 2017 at 5:46 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:17AM -0700, Dan Williams wrote:
>> Move xfs_break_layouts() to its own compilation unit so that it can be
>> used for both pnfs layouts and MAP_DIRECT mappings.
> .....
>> diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
>> index b587cb99b2b7..4135b2482697 100644
>> --- a/fs/xfs/xfs_pnfs.h
>> +++ b/fs/xfs/xfs_pnfs.h
>> @@ -1,19 +1,13 @@
>> #ifndef _XFS_PNFS_H
>> #define _XFS_PNFS_H 1
>>
>> +#include "xfs_layout.h"
>> +
>
> I missed this the first time through - we try not to put includes
> in header files, and instead make sure each C file has all the
> includes they require. Can you move this to all the C files that
> need layouts and remove the include of the xfs_pnfs.h include from
> them?
Sure, will do.
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 04/14] xfs: prepare xfs_break_layouts() for reuse with MAP_DIRECT
@ 2017-10-11 2:12 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-nvdimm, Jan Kara, Darrick J. Wong, linux-rdma, Linux API,
iommu, linux-xfs, Linux MM, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
On Tue, Oct 10, 2017 at 5:46 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:17AM -0700, Dan Williams wrote:
>> Move xfs_break_layouts() to its own compilation unit so that it can be
>> used for both pnfs layouts and MAP_DIRECT mappings.
> .....
>> diff --git a/fs/xfs/xfs_pnfs.h b/fs/xfs/xfs_pnfs.h
>> index b587cb99b2b7..4135b2482697 100644
>> --- a/fs/xfs/xfs_pnfs.h
>> +++ b/fs/xfs/xfs_pnfs.h
>> @@ -1,19 +1,13 @@
>> #ifndef _XFS_PNFS_H
>> #define _XFS_PNFS_H 1
>>
>> +#include "xfs_layout.h"
>> +
>
> I missed this the first time through - we try not to put includes
> in header files, and instead make sure each C file has all the
> includes they require. Can you move this to all the C files that
> need layouts and remove the include of the xfs_pnfs.h include from
> them?
Sure, will do.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 05/14] fs, xfs, iomap: introduce iomap_can_allocate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
linux-xfs, linux-mm, iommu, linux-fsdevel, Christoph Hellwig
In preparation for using FL_LAYOUT leases to allow coordination between
the kernel and processes doing userspace flushes / RDMA with DAX
mappings, add this helper that can be used to detect when block-map
updates are not allowed.
This is targeted to be used in an ->iomap_begin() implementation where
we may have various filesystem locks held and can not synchronously wait
for any FL_LAYOUT leases to be released. In particular an iomap mmap
fault handler running under mmap_sem can not unlock that semaphore and
wait for these leases to be unlocked. Instead, this signals the lease
holder(s) that a break is requested and immediately returns with an
error.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_iomap.c | 3 +++
fs/xfs/xfs_layout.c | 5 ++++-
include/linux/iomap.h | 10 ++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index a1909bc064e9..b3cda11e9515 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -1052,6 +1052,9 @@ xfs_file_iomap_begin(
error = -EAGAIN;
goto out_unlock;
}
+ error = iomap_can_allocate(inode);
+ if (error)
+ goto out_unlock;
/*
* We cap the maximum length we map here to MAX_WRITEBACK_PAGES
* pages to keep the chunks of work done where somewhat symmetric
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
index 71d95e1a910a..88c533bf5b7c 100644
--- a/fs/xfs/xfs_layout.c
+++ b/fs/xfs/xfs_layout.c
@@ -19,7 +19,10 @@
* about exposing unallocated blocks but just want to provide basic
* synchronization between a local writer and pNFS clients. mmap writes would
* also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
+ * rules in the page fault path all we can do is start the lease break
+ * timeout. See usage of iomap_can_allocate in xfs_file_iomap_begin to
+ * prevent write-faults from allocating blocks or performing extent
+ * conversion.
*/
int
xfs_break_layouts(
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index f64dc6ce5161..e24b4e81d41a 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -2,6 +2,7 @@
#define LINUX_IOMAP_H 1
#include <linux/types.h>
+#include <linux/fs.h>
struct fiemap_extent_info;
struct inode;
@@ -88,6 +89,15 @@ loff_t iomap_seek_hole(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
loff_t iomap_seek_data(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
+/*
+ * Check if there are any file layout leases preventing block map
+ * changes and if so start the lease break process, but do not wait for
+ * it to complete (return -EWOULDBLOCK);
+ */
+static inline int iomap_can_allocate(struct inode *inode)
+{
+ return break_layout(inode, false);
+}
/*
* Flags for direct I/O ->end_io:
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 05/14] fs, xfs, iomap: introduce iomap_can_allocate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
In preparation for using FL_LAYOUT leases to allow coordination between
the kernel and processes doing userspace flushes / RDMA with DAX
mappings, add this helper that can be used to detect when block-map
updates are not allowed.
This is targeted to be used in an ->iomap_begin() implementation where
we may have various filesystem locks held and can not synchronously wait
for any FL_LAYOUT leases to be released. In particular an iomap mmap
fault handler running under mmap_sem can not unlock that semaphore and
wait for these leases to be unlocked. Instead, this signals the lease
holder(s) that a break is requested and immediately returns with an
error.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_iomap.c | 3 +++
fs/xfs/xfs_layout.c | 5 ++++-
include/linux/iomap.h | 10 ++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index a1909bc064e9..b3cda11e9515 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -1052,6 +1052,9 @@ xfs_file_iomap_begin(
error = -EAGAIN;
goto out_unlock;
}
+ error = iomap_can_allocate(inode);
+ if (error)
+ goto out_unlock;
/*
* We cap the maximum length we map here to MAX_WRITEBACK_PAGES
* pages to keep the chunks of work done where somewhat symmetric
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
index 71d95e1a910a..88c533bf5b7c 100644
--- a/fs/xfs/xfs_layout.c
+++ b/fs/xfs/xfs_layout.c
@@ -19,7 +19,10 @@
* about exposing unallocated blocks but just want to provide basic
* synchronization between a local writer and pNFS clients. mmap writes would
* also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
+ * rules in the page fault path all we can do is start the lease break
+ * timeout. See usage of iomap_can_allocate in xfs_file_iomap_begin to
+ * prevent write-faults from allocating blocks or performing extent
+ * conversion.
*/
int
xfs_break_layouts(
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index f64dc6ce5161..e24b4e81d41a 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -2,6 +2,7 @@
#define LINUX_IOMAP_H 1
#include <linux/types.h>
+#include <linux/fs.h>
struct fiemap_extent_info;
struct inode;
@@ -88,6 +89,15 @@ loff_t iomap_seek_hole(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
loff_t iomap_seek_data(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
+/*
+ * Check if there are any file layout leases preventing block map
+ * changes and if so start the lease break process, but do not wait for
+ * it to complete (return -EWOULDBLOCK);
+ */
+static inline int iomap_can_allocate(struct inode *inode)
+{
+ return break_layout(inode, false);
+}
/*
* Flags for direct I/O ->end_io:
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 05/14] fs, xfs, iomap: introduce iomap_can_allocate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Darrick J. Wong, linux-rdma, linux-api, Dave Chinner,
iommu, linux-xfs, linux-mm, Jeff Moyer, linux-fsdevel,
Ross Zwisler, Christoph Hellwig
In preparation for using FL_LAYOUT leases to allow coordination between
the kernel and processes doing userspace flushes / RDMA with DAX
mappings, add this helper that can be used to detect when block-map
updates are not allowed.
This is targeted to be used in an ->iomap_begin() implementation where
we may have various filesystem locks held and can not synchronously wait
for any FL_LAYOUT leases to be released. In particular an iomap mmap
fault handler running under mmap_sem can not unlock that semaphore and
wait for these leases to be unlocked. Instead, this signals the lease
holder(s) that a break is requested and immediately returns with an
error.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Suggested-by: Dave Chinner <david@fromorbit.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_iomap.c | 3 +++
fs/xfs/xfs_layout.c | 5 ++++-
include/linux/iomap.h | 10 ++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index a1909bc064e9..b3cda11e9515 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -1052,6 +1052,9 @@ xfs_file_iomap_begin(
error = -EAGAIN;
goto out_unlock;
}
+ error = iomap_can_allocate(inode);
+ if (error)
+ goto out_unlock;
/*
* We cap the maximum length we map here to MAX_WRITEBACK_PAGES
* pages to keep the chunks of work done where somewhat symmetric
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
index 71d95e1a910a..88c533bf5b7c 100644
--- a/fs/xfs/xfs_layout.c
+++ b/fs/xfs/xfs_layout.c
@@ -19,7 +19,10 @@
* about exposing unallocated blocks but just want to provide basic
* synchronization between a local writer and pNFS clients. mmap writes would
* also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
+ * rules in the page fault path all we can do is start the lease break
+ * timeout. See usage of iomap_can_allocate in xfs_file_iomap_begin to
+ * prevent write-faults from allocating blocks or performing extent
+ * conversion.
*/
int
xfs_break_layouts(
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index f64dc6ce5161..e24b4e81d41a 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -2,6 +2,7 @@
#define LINUX_IOMAP_H 1
#include <linux/types.h>
+#include <linux/fs.h>
struct fiemap_extent_info;
struct inode;
@@ -88,6 +89,15 @@ loff_t iomap_seek_hole(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
loff_t iomap_seek_data(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
+/*
+ * Check if there are any file layout leases preventing block map
+ * changes and if so start the lease break process, but do not wait for
+ * it to complete (return -EWOULDBLOCK);
+ */
+static inline int iomap_can_allocate(struct inode *inode)
+{
+ return break_layout(inode, false);
+}
/*
* Flags for direct I/O ->end_io:
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 05/14] fs, xfs, iomap: introduce iomap_can_allocate()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: Jan Kara, Darrick J. Wong, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Dave Chinner,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Christoph Hellwig
In preparation for using FL_LAYOUT leases to allow coordination between
the kernel and processes doing userspace flushes / RDMA with DAX
mappings, add this helper that can be used to detect when block-map
updates are not allowed.
This is targeted to be used in an ->iomap_begin() implementation where
we may have various filesystem locks held and can not synchronously wait
for any FL_LAYOUT leases to be released. In particular an iomap mmap
fault handler running under mmap_sem can not unlock that semaphore and
wait for these leases to be unlocked. Instead, this signals the lease
holder(s) that a break is requested and immediately returns with an
error.
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Cc: Jeff Moyer <jmoyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Cc: "Darrick J. Wong" <darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Suggested-by: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
fs/xfs/xfs_iomap.c | 3 +++
fs/xfs/xfs_layout.c | 5 ++++-
include/linux/iomap.h | 10 ++++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index a1909bc064e9..b3cda11e9515 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -1052,6 +1052,9 @@ xfs_file_iomap_begin(
error = -EAGAIN;
goto out_unlock;
}
+ error = iomap_can_allocate(inode);
+ if (error)
+ goto out_unlock;
/*
* We cap the maximum length we map here to MAX_WRITEBACK_PAGES
* pages to keep the chunks of work done where somewhat symmetric
diff --git a/fs/xfs/xfs_layout.c b/fs/xfs/xfs_layout.c
index 71d95e1a910a..88c533bf5b7c 100644
--- a/fs/xfs/xfs_layout.c
+++ b/fs/xfs/xfs_layout.c
@@ -19,7 +19,10 @@
* about exposing unallocated blocks but just want to provide basic
* synchronization between a local writer and pNFS clients. mmap writes would
* also benefit from this sort of synchronization, but due to the tricky locking
- * rules in the page fault path we don't bother.
+ * rules in the page fault path all we can do is start the lease break
+ * timeout. See usage of iomap_can_allocate in xfs_file_iomap_begin to
+ * prevent write-faults from allocating blocks or performing extent
+ * conversion.
*/
int
xfs_break_layouts(
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index f64dc6ce5161..e24b4e81d41a 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -2,6 +2,7 @@
#define LINUX_IOMAP_H 1
#include <linux/types.h>
+#include <linux/fs.h>
struct fiemap_extent_info;
struct inode;
@@ -88,6 +89,15 @@ loff_t iomap_seek_hole(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
loff_t iomap_seek_data(struct inode *inode, loff_t offset,
const struct iomap_ops *ops);
+/*
+ * Check if there are any file layout leases preventing block map
+ * changes and if so start the lease break process, but do not wait for
+ * it to complete (return -EWOULDBLOCK);
+ */
+static inline int iomap_can_allocate(struct inode *inode)
+{
+ return break_layout(inode, false);
+}
/*
* Flags for direct I/O ->end_io:
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: J. Bruce Fields, Jan Kara, Arnd Bergmann, Darrick J. Wong,
linux-rdma, linux-api, Dave Chinner, linux-xfs, linux-mm, iommu,
Alexander Viro, linux-fsdevel, Jeff Layton, Christoph Hellwig
MAP_DIRECT is an mmap(2) flag with the following semantics:
MAP_DIRECT
When specified with MAP_SHARED_VALIDATE, sets up a file lease with the
same lifetime as the mapping. Unlike a typical F_RDLCK lease this lease
is broken when a "lease breaker" attempts to write(2), change the block
map (fallocate), or change the size of the file. Otherwise the mechanism
of a lease break is identical to the typical lease break case where the
lease needs to be removed (munmap) within the number of seconds
specified by /proc/sys/fs/lease-break-time. If the lease holder fails to
remove the lease in time the kernel will invalidate the mapping and
force all future accesses to the mapping to trigger SIGBUS.
In addition to lease break timeouts causing faults in the mapping to
result in SIGBUS, other states of the file will trigger SIGBUS at fault
time:
* The fault would trigger the filesystem to allocate blocks
* The fault would trigger the filesystem to perform extent conversion
In other words, MAP_DIRECT expects and enforces a fully allocated file
where faults can be satisfied without modifying block map metadata.
An unprivileged process may establish a MAP_DIRECT mapping on a file
whose UID (owner) matches the filesystem UID of the process. A process
with the CAP_LEASE capability may establish a MAP_DIRECT mapping on
arbitrary files
ERRORS
EACCES Beyond the typical mmap(2) conditions that trigger EACCES
MAP_DIRECT also requires the permission to set a file lease.
EOPNOTSUPP The filesystem explicitly does not support the flag
EPERM The file does not permit MAP_DIRECT mappings. Potential reasons
are that DAX access is not available or the file has reflink extents.
SIGBUS Attempted to write a MAP_DIRECT mapping at a file offset that
might require block-map updates, or the lease timed out and the
kernel invalidated the mapping.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 2 -
fs/xfs/xfs_file.c | 103 ++++++++++++++++++++++++++++++++++++++-
include/linux/mman.h | 3 +
include/uapi/asm-generic/mman.h | 1
4 files changed, 106 insertions(+), 3 deletions(-)
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index f62fc6629abb..f8765653a438 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -112,4 +112,4 @@ config XFS_ASSERT_FATAL
config XFS_LAYOUT
def_bool y
- depends on EXPORTFS_BLOCK_OPS
+ depends on EXPORTFS_BLOCK_OPS || FS_DAX
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index ebdd0bd2b261..4bee027c9366 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -40,12 +40,22 @@
#include "xfs_iomap.h"
#include "xfs_reflink.h"
+#include <linux/mman.h>
#include <linux/dcache.h>
#include <linux/falloc.h>
#include <linux/pagevec.h>
+#include <linux/mapdirect.h>
#include <linux/backing-dev.h>
static const struct vm_operations_struct xfs_file_vm_ops;
+static const struct vm_operations_struct xfs_file_vm_direct_ops;
+
+static bool
+xfs_vma_is_direct(
+ struct vm_area_struct *vma)
+{
+ return vma->vm_ops == &xfs_file_vm_direct_ops;
+}
/*
* Clear the specified ranges to zero through either the pagecache or DAX.
@@ -1009,6 +1019,22 @@ xfs_file_llseek(
}
/*
+ * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
+ * valid. See map_direct_invalidate.
+ */
+static int
+xfs_can_fault_direct(
+ struct vm_area_struct *vma)
+{
+ if (!xfs_vma_is_direct(vma))
+ return 0;
+
+ if (!test_map_direct_valid(vma->vm_private_data))
+ return VM_FAULT_SIGBUS;
+ return 0;
+}
+
+/*
* Locking for serialisation of IO during page faults. This results in a lock
* ordering of:
*
@@ -1024,7 +1050,8 @@ __xfs_filemap_fault(
enum page_entry_size pe_size,
bool write_fault)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct vm_area_struct *vma = vmf->vma;
+ struct inode *inode = file_inode(vma->vm_file);
struct xfs_inode *ip = XFS_I(inode);
int ret;
@@ -1032,10 +1059,14 @@ __xfs_filemap_fault(
if (write_fault) {
sb_start_pagefault(inode->i_sb);
- file_update_time(vmf->vma->vm_file);
+ file_update_time(vma->vm_file);
}
xfs_ilock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
+ ret = xfs_can_fault_direct(vma);
+ if (ret)
+ goto out_unlock;
+
if (IS_DAX(inode)) {
ret = dax_iomap_fault(vmf, pe_size, &xfs_iomap_ops);
} else {
@@ -1044,6 +1075,8 @@ __xfs_filemap_fault(
else
ret = filemap_fault(vmf);
}
+
+out_unlock:
xfs_iunlock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
if (write_fault)
@@ -1115,6 +1148,17 @@ xfs_filemap_pfn_mkwrite(
}
+static const struct vm_operations_struct xfs_file_vm_direct_ops = {
+ .fault = xfs_filemap_fault,
+ .huge_fault = xfs_filemap_huge_fault,
+ .map_pages = filemap_map_pages,
+ .page_mkwrite = xfs_filemap_page_mkwrite,
+ .pfn_mkwrite = xfs_filemap_pfn_mkwrite,
+
+ .open = generic_map_direct_open,
+ .close = generic_map_direct_close,
+};
+
static const struct vm_operations_struct xfs_file_vm_ops = {
.fault = xfs_filemap_fault,
.huge_fault = xfs_filemap_huge_fault,
@@ -1135,6 +1179,60 @@ xfs_file_mmap(
return 0;
}
+static int
+xfs_file_mmap_direct(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ int fd)
+{
+ struct inode *inode = file_inode(filp);
+ struct xfs_inode *ip = XFS_I(inode);
+ struct map_direct_state *mds;
+
+ /*
+ * Not permitted to set up MAP_DIRECT mapping over reflinked or
+ * non-DAX extents since reflink may cause block moves /
+ * copy-on-write, and non-DAX is by definition always indirect
+ * through the page cache.
+ */
+ if (xfs_is_reflink_inode(ip))
+ return -EPERM;
+ if (!IS_DAX(inode))
+ return -EPERM;
+
+ mds = map_direct_register(fd, vma);
+ if (IS_ERR(mds))
+ return PTR_ERR(mds);
+
+ file_accessed(filp);
+ vma->vm_ops = &xfs_file_vm_direct_ops;
+ vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+
+ /*
+ * generic_map_direct_{open,close} expect ->vm_private_data is
+ * set to the result of map_direct_register
+ */
+ vma->vm_private_data = mds;
+ return 0;
+}
+
+#define XFS_MAP_SUPPORTED (LEGACY_MAP_MASK | MAP_DIRECT)
+
+static int
+xfs_file_mmap_validate(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ unsigned long map_flags,
+ int fd)
+{
+ if (map_flags & ~(XFS_MAP_SUPPORTED))
+ return -EOPNOTSUPP;
+
+ if ((map_flags & MAP_DIRECT) == 0)
+ return xfs_file_mmap(filp, vma);
+ return xfs_file_mmap_direct(filp, vma, fd);
+}
+
const struct file_operations xfs_file_operations = {
.llseek = xfs_file_llseek,
.read_iter = xfs_file_read_iter,
@@ -1146,6 +1244,7 @@ const struct file_operations xfs_file_operations = {
.compat_ioctl = xfs_file_compat_ioctl,
#endif
.mmap = xfs_file_mmap,
+ .mmap_validate = xfs_file_mmap_validate,
.open = xfs_file_open,
.release = xfs_file_release,
.fsync = xfs_file_fsync,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index 94b63b4d71ff..fab393a9dda9 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -20,6 +20,9 @@
#ifndef MAP_HUGE_1GB
#define MAP_HUGE_1GB 0
#endif
+#ifndef MAP_DIRECT
+#define MAP_DIRECT 0
+#endif
#ifndef MAP_UNINITIALIZED
#define MAP_UNINITIALIZED 0
#endif
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index 7162cd4cca73..c916f22008e0 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -12,6 +12,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_DIRECT 0x80000 /* leased block map (layout) for DAX */
/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Arnd Bergmann, Darrick J. Wong, linux-rdma,
linux-api, Dave Chinner, iommu, Christoph Hellwig,
J. Bruce Fields, linux-mm, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
MAP_DIRECT is an mmap(2) flag with the following semantics:
MAP_DIRECT
When specified with MAP_SHARED_VALIDATE, sets up a file lease with the
same lifetime as the mapping. Unlike a typical F_RDLCK lease this lease
is broken when a "lease breaker" attempts to write(2), change the block
map (fallocate), or change the size of the file. Otherwise the mechanism
of a lease break is identical to the typical lease break case where the
lease needs to be removed (munmap) within the number of seconds
specified by /proc/sys/fs/lease-break-time. If the lease holder fails to
remove the lease in time the kernel will invalidate the mapping and
force all future accesses to the mapping to trigger SIGBUS.
In addition to lease break timeouts causing faults in the mapping to
result in SIGBUS, other states of the file will trigger SIGBUS at fault
time:
* The fault would trigger the filesystem to allocate blocks
* The fault would trigger the filesystem to perform extent conversion
In other words, MAP_DIRECT expects and enforces a fully allocated file
where faults can be satisfied without modifying block map metadata.
An unprivileged process may establish a MAP_DIRECT mapping on a file
whose UID (owner) matches the filesystem UID of the process. A process
with the CAP_LEASE capability may establish a MAP_DIRECT mapping on
arbitrary files
ERRORS
EACCES Beyond the typical mmap(2) conditions that trigger EACCES
MAP_DIRECT also requires the permission to set a file lease.
EOPNOTSUPP The filesystem explicitly does not support the flag
EPERM The file does not permit MAP_DIRECT mappings. Potential reasons
are that DAX access is not available or the file has reflink extents.
SIGBUS Attempted to write a MAP_DIRECT mapping at a file offset that
might require block-map updates, or the lease timed out and the
kernel invalidated the mapping.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 2 -
fs/xfs/xfs_file.c | 103 ++++++++++++++++++++++++++++++++++++++-
include/linux/mman.h | 3 +
include/uapi/asm-generic/mman.h | 1
4 files changed, 106 insertions(+), 3 deletions(-)
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index f62fc6629abb..f8765653a438 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -112,4 +112,4 @@ config XFS_ASSERT_FATAL
config XFS_LAYOUT
def_bool y
- depends on EXPORTFS_BLOCK_OPS
+ depends on EXPORTFS_BLOCK_OPS || FS_DAX
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index ebdd0bd2b261..4bee027c9366 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -40,12 +40,22 @@
#include "xfs_iomap.h"
#include "xfs_reflink.h"
+#include <linux/mman.h>
#include <linux/dcache.h>
#include <linux/falloc.h>
#include <linux/pagevec.h>
+#include <linux/mapdirect.h>
#include <linux/backing-dev.h>
static const struct vm_operations_struct xfs_file_vm_ops;
+static const struct vm_operations_struct xfs_file_vm_direct_ops;
+
+static bool
+xfs_vma_is_direct(
+ struct vm_area_struct *vma)
+{
+ return vma->vm_ops == &xfs_file_vm_direct_ops;
+}
/*
* Clear the specified ranges to zero through either the pagecache or DAX.
@@ -1009,6 +1019,22 @@ xfs_file_llseek(
}
/*
+ * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
+ * valid. See map_direct_invalidate.
+ */
+static int
+xfs_can_fault_direct(
+ struct vm_area_struct *vma)
+{
+ if (!xfs_vma_is_direct(vma))
+ return 0;
+
+ if (!test_map_direct_valid(vma->vm_private_data))
+ return VM_FAULT_SIGBUS;
+ return 0;
+}
+
+/*
* Locking for serialisation of IO during page faults. This results in a lock
* ordering of:
*
@@ -1024,7 +1050,8 @@ __xfs_filemap_fault(
enum page_entry_size pe_size,
bool write_fault)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct vm_area_struct *vma = vmf->vma;
+ struct inode *inode = file_inode(vma->vm_file);
struct xfs_inode *ip = XFS_I(inode);
int ret;
@@ -1032,10 +1059,14 @@ __xfs_filemap_fault(
if (write_fault) {
sb_start_pagefault(inode->i_sb);
- file_update_time(vmf->vma->vm_file);
+ file_update_time(vma->vm_file);
}
xfs_ilock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
+ ret = xfs_can_fault_direct(vma);
+ if (ret)
+ goto out_unlock;
+
if (IS_DAX(inode)) {
ret = dax_iomap_fault(vmf, pe_size, &xfs_iomap_ops);
} else {
@@ -1044,6 +1075,8 @@ __xfs_filemap_fault(
else
ret = filemap_fault(vmf);
}
+
+out_unlock:
xfs_iunlock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
if (write_fault)
@@ -1115,6 +1148,17 @@ xfs_filemap_pfn_mkwrite(
}
+static const struct vm_operations_struct xfs_file_vm_direct_ops = {
+ .fault = xfs_filemap_fault,
+ .huge_fault = xfs_filemap_huge_fault,
+ .map_pages = filemap_map_pages,
+ .page_mkwrite = xfs_filemap_page_mkwrite,
+ .pfn_mkwrite = xfs_filemap_pfn_mkwrite,
+
+ .open = generic_map_direct_open,
+ .close = generic_map_direct_close,
+};
+
static const struct vm_operations_struct xfs_file_vm_ops = {
.fault = xfs_filemap_fault,
.huge_fault = xfs_filemap_huge_fault,
@@ -1135,6 +1179,60 @@ xfs_file_mmap(
return 0;
}
+static int
+xfs_file_mmap_direct(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ int fd)
+{
+ struct inode *inode = file_inode(filp);
+ struct xfs_inode *ip = XFS_I(inode);
+ struct map_direct_state *mds;
+
+ /*
+ * Not permitted to set up MAP_DIRECT mapping over reflinked or
+ * non-DAX extents since reflink may cause block moves /
+ * copy-on-write, and non-DAX is by definition always indirect
+ * through the page cache.
+ */
+ if (xfs_is_reflink_inode(ip))
+ return -EPERM;
+ if (!IS_DAX(inode))
+ return -EPERM;
+
+ mds = map_direct_register(fd, vma);
+ if (IS_ERR(mds))
+ return PTR_ERR(mds);
+
+ file_accessed(filp);
+ vma->vm_ops = &xfs_file_vm_direct_ops;
+ vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+
+ /*
+ * generic_map_direct_{open,close} expect ->vm_private_data is
+ * set to the result of map_direct_register
+ */
+ vma->vm_private_data = mds;
+ return 0;
+}
+
+#define XFS_MAP_SUPPORTED (LEGACY_MAP_MASK | MAP_DIRECT)
+
+static int
+xfs_file_mmap_validate(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ unsigned long map_flags,
+ int fd)
+{
+ if (map_flags & ~(XFS_MAP_SUPPORTED))
+ return -EOPNOTSUPP;
+
+ if ((map_flags & MAP_DIRECT) == 0)
+ return xfs_file_mmap(filp, vma);
+ return xfs_file_mmap_direct(filp, vma, fd);
+}
+
const struct file_operations xfs_file_operations = {
.llseek = xfs_file_llseek,
.read_iter = xfs_file_read_iter,
@@ -1146,6 +1244,7 @@ const struct file_operations xfs_file_operations = {
.compat_ioctl = xfs_file_compat_ioctl,
#endif
.mmap = xfs_file_mmap,
+ .mmap_validate = xfs_file_mmap_validate,
.open = xfs_file_open,
.release = xfs_file_release,
.fsync = xfs_file_fsync,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index 94b63b4d71ff..fab393a9dda9 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -20,6 +20,9 @@
#ifndef MAP_HUGE_1GB
#define MAP_HUGE_1GB 0
#endif
+#ifndef MAP_DIRECT
+#define MAP_DIRECT 0
+#endif
#ifndef MAP_UNINITIALIZED
#define MAP_UNINITIALIZED 0
#endif
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index 7162cd4cca73..c916f22008e0 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -12,6 +12,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_DIRECT 0x80000 /* leased block map (layout) for DAX */
/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Arnd Bergmann, Darrick J. Wong, linux-rdma,
linux-api, Dave Chinner, iommu, Christoph Hellwig,
J. Bruce Fields, linux-mm, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
MAP_DIRECT is an mmap(2) flag with the following semantics:
MAP_DIRECT
When specified with MAP_SHARED_VALIDATE, sets up a file lease with the
same lifetime as the mapping. Unlike a typical F_RDLCK lease this lease
is broken when a "lease breaker" attempts to write(2), change the block
map (fallocate), or change the size of the file. Otherwise the mechanism
of a lease break is identical to the typical lease break case where the
lease needs to be removed (munmap) within the number of seconds
specified by /proc/sys/fs/lease-break-time. If the lease holder fails to
remove the lease in time the kernel will invalidate the mapping and
force all future accesses to the mapping to trigger SIGBUS.
In addition to lease break timeouts causing faults in the mapping to
result in SIGBUS, other states of the file will trigger SIGBUS at fault
time:
* The fault would trigger the filesystem to allocate blocks
* The fault would trigger the filesystem to perform extent conversion
In other words, MAP_DIRECT expects and enforces a fully allocated file
where faults can be satisfied without modifying block map metadata.
An unprivileged process may establish a MAP_DIRECT mapping on a file
whose UID (owner) matches the filesystem UID of the process. A process
with the CAP_LEASE capability may establish a MAP_DIRECT mapping on
arbitrary files
ERRORS
EACCES Beyond the typical mmap(2) conditions that trigger EACCES
MAP_DIRECT also requires the permission to set a file lease.
EOPNOTSUPP The filesystem explicitly does not support the flag
EPERM The file does not permit MAP_DIRECT mappings. Potential reasons
are that DAX access is not available or the file has reflink extents.
SIGBUS Attempted to write a MAP_DIRECT mapping at a file offset that
might require block-map updates, or the lease timed out and the
kernel invalidated the mapping.
Cc: Jan Kara <jack@suse.cz>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/Kconfig | 2 -
fs/xfs/xfs_file.c | 103 ++++++++++++++++++++++++++++++++++++++-
include/linux/mman.h | 3 +
include/uapi/asm-generic/mman.h | 1
4 files changed, 106 insertions(+), 3 deletions(-)
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index f62fc6629abb..f8765653a438 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -112,4 +112,4 @@ config XFS_ASSERT_FATAL
config XFS_LAYOUT
def_bool y
- depends on EXPORTFS_BLOCK_OPS
+ depends on EXPORTFS_BLOCK_OPS || FS_DAX
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index ebdd0bd2b261..4bee027c9366 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -40,12 +40,22 @@
#include "xfs_iomap.h"
#include "xfs_reflink.h"
+#include <linux/mman.h>
#include <linux/dcache.h>
#include <linux/falloc.h>
#include <linux/pagevec.h>
+#include <linux/mapdirect.h>
#include <linux/backing-dev.h>
static const struct vm_operations_struct xfs_file_vm_ops;
+static const struct vm_operations_struct xfs_file_vm_direct_ops;
+
+static bool
+xfs_vma_is_direct(
+ struct vm_area_struct *vma)
+{
+ return vma->vm_ops == &xfs_file_vm_direct_ops;
+}
/*
* Clear the specified ranges to zero through either the pagecache or DAX.
@@ -1009,6 +1019,22 @@ xfs_file_llseek(
}
/*
+ * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
+ * valid. See map_direct_invalidate.
+ */
+static int
+xfs_can_fault_direct(
+ struct vm_area_struct *vma)
+{
+ if (!xfs_vma_is_direct(vma))
+ return 0;
+
+ if (!test_map_direct_valid(vma->vm_private_data))
+ return VM_FAULT_SIGBUS;
+ return 0;
+}
+
+/*
* Locking for serialisation of IO during page faults. This results in a lock
* ordering of:
*
@@ -1024,7 +1050,8 @@ __xfs_filemap_fault(
enum page_entry_size pe_size,
bool write_fault)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct vm_area_struct *vma = vmf->vma;
+ struct inode *inode = file_inode(vma->vm_file);
struct xfs_inode *ip = XFS_I(inode);
int ret;
@@ -1032,10 +1059,14 @@ __xfs_filemap_fault(
if (write_fault) {
sb_start_pagefault(inode->i_sb);
- file_update_time(vmf->vma->vm_file);
+ file_update_time(vma->vm_file);
}
xfs_ilock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
+ ret = xfs_can_fault_direct(vma);
+ if (ret)
+ goto out_unlock;
+
if (IS_DAX(inode)) {
ret = dax_iomap_fault(vmf, pe_size, &xfs_iomap_ops);
} else {
@@ -1044,6 +1075,8 @@ __xfs_filemap_fault(
else
ret = filemap_fault(vmf);
}
+
+out_unlock:
xfs_iunlock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
if (write_fault)
@@ -1115,6 +1148,17 @@ xfs_filemap_pfn_mkwrite(
}
+static const struct vm_operations_struct xfs_file_vm_direct_ops = {
+ .fault = xfs_filemap_fault,
+ .huge_fault = xfs_filemap_huge_fault,
+ .map_pages = filemap_map_pages,
+ .page_mkwrite = xfs_filemap_page_mkwrite,
+ .pfn_mkwrite = xfs_filemap_pfn_mkwrite,
+
+ .open = generic_map_direct_open,
+ .close = generic_map_direct_close,
+};
+
static const struct vm_operations_struct xfs_file_vm_ops = {
.fault = xfs_filemap_fault,
.huge_fault = xfs_filemap_huge_fault,
@@ -1135,6 +1179,60 @@ xfs_file_mmap(
return 0;
}
+static int
+xfs_file_mmap_direct(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ int fd)
+{
+ struct inode *inode = file_inode(filp);
+ struct xfs_inode *ip = XFS_I(inode);
+ struct map_direct_state *mds;
+
+ /*
+ * Not permitted to set up MAP_DIRECT mapping over reflinked or
+ * non-DAX extents since reflink may cause block moves /
+ * copy-on-write, and non-DAX is by definition always indirect
+ * through the page cache.
+ */
+ if (xfs_is_reflink_inode(ip))
+ return -EPERM;
+ if (!IS_DAX(inode))
+ return -EPERM;
+
+ mds = map_direct_register(fd, vma);
+ if (IS_ERR(mds))
+ return PTR_ERR(mds);
+
+ file_accessed(filp);
+ vma->vm_ops = &xfs_file_vm_direct_ops;
+ vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+
+ /*
+ * generic_map_direct_{open,close} expect ->vm_private_data is
+ * set to the result of map_direct_register
+ */
+ vma->vm_private_data = mds;
+ return 0;
+}
+
+#define XFS_MAP_SUPPORTED (LEGACY_MAP_MASK | MAP_DIRECT)
+
+static int
+xfs_file_mmap_validate(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ unsigned long map_flags,
+ int fd)
+{
+ if (map_flags & ~(XFS_MAP_SUPPORTED))
+ return -EOPNOTSUPP;
+
+ if ((map_flags & MAP_DIRECT) == 0)
+ return xfs_file_mmap(filp, vma);
+ return xfs_file_mmap_direct(filp, vma, fd);
+}
+
const struct file_operations xfs_file_operations = {
.llseek = xfs_file_llseek,
.read_iter = xfs_file_read_iter,
@@ -1146,6 +1244,7 @@ const struct file_operations xfs_file_operations = {
.compat_ioctl = xfs_file_compat_ioctl,
#endif
.mmap = xfs_file_mmap,
+ .mmap_validate = xfs_file_mmap_validate,
.open = xfs_file_open,
.release = xfs_file_release,
.fsync = xfs_file_fsync,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index 94b63b4d71ff..fab393a9dda9 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -20,6 +20,9 @@
#ifndef MAP_HUGE_1GB
#define MAP_HUGE_1GB 0
#endif
+#ifndef MAP_DIRECT
+#define MAP_DIRECT 0
+#endif
#ifndef MAP_UNINITIALIZED
#define MAP_UNINITIALIZED 0
#endif
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index 7162cd4cca73..c916f22008e0 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -12,6 +12,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_DIRECT 0x80000 /* leased block map (layout) for DAX */
/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: linux-xfs-u79uwXL29TY76Z2rM5mHXA, Jan Kara, Arnd Bergmann,
Darrick J. Wong, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Dave Chinner,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Christoph Hellwig, J. Bruce Fields,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg, Jeff Moyer, Alexander Viro,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Jeff Layton, Ross Zwisler
MAP_DIRECT is an mmap(2) flag with the following semantics:
MAP_DIRECT
When specified with MAP_SHARED_VALIDATE, sets up a file lease with the
same lifetime as the mapping. Unlike a typical F_RDLCK lease this lease
is broken when a "lease breaker" attempts to write(2), change the block
map (fallocate), or change the size of the file. Otherwise the mechanism
of a lease break is identical to the typical lease break case where the
lease needs to be removed (munmap) within the number of seconds
specified by /proc/sys/fs/lease-break-time. If the lease holder fails to
remove the lease in time the kernel will invalidate the mapping and
force all future accesses to the mapping to trigger SIGBUS.
In addition to lease break timeouts causing faults in the mapping to
result in SIGBUS, other states of the file will trigger SIGBUS at fault
time:
* The fault would trigger the filesystem to allocate blocks
* The fault would trigger the filesystem to perform extent conversion
In other words, MAP_DIRECT expects and enforces a fully allocated file
where faults can be satisfied without modifying block map metadata.
An unprivileged process may establish a MAP_DIRECT mapping on a file
whose UID (owner) matches the filesystem UID of the process. A process
with the CAP_LEASE capability may establish a MAP_DIRECT mapping on
arbitrary files
ERRORS
EACCES Beyond the typical mmap(2) conditions that trigger EACCES
MAP_DIRECT also requires the permission to set a file lease.
EOPNOTSUPP The filesystem explicitly does not support the flag
EPERM The file does not permit MAP_DIRECT mappings. Potential reasons
are that DAX access is not available or the file has reflink extents.
SIGBUS Attempted to write a MAP_DIRECT mapping at a file offset that
might require block-map updates, or the lease timed out and the
kernel invalidated the mapping.
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Cc: Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>
Cc: Jeff Moyer <jmoyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Cc: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Cc: Alexander Viro <viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn@public.gmane.org>
Cc: "Darrick J. Wong" <darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: Jeff Layton <jlayton-vpEMnDpepFuMZCB2o+C8xQ@public.gmane.org>
Cc: "J. Bruce Fields" <bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
fs/xfs/Kconfig | 2 -
fs/xfs/xfs_file.c | 103 ++++++++++++++++++++++++++++++++++++++-
include/linux/mman.h | 3 +
include/uapi/asm-generic/mman.h | 1
4 files changed, 106 insertions(+), 3 deletions(-)
diff --git a/fs/xfs/Kconfig b/fs/xfs/Kconfig
index f62fc6629abb..f8765653a438 100644
--- a/fs/xfs/Kconfig
+++ b/fs/xfs/Kconfig
@@ -112,4 +112,4 @@ config XFS_ASSERT_FATAL
config XFS_LAYOUT
def_bool y
- depends on EXPORTFS_BLOCK_OPS
+ depends on EXPORTFS_BLOCK_OPS || FS_DAX
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index ebdd0bd2b261..4bee027c9366 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -40,12 +40,22 @@
#include "xfs_iomap.h"
#include "xfs_reflink.h"
+#include <linux/mman.h>
#include <linux/dcache.h>
#include <linux/falloc.h>
#include <linux/pagevec.h>
+#include <linux/mapdirect.h>
#include <linux/backing-dev.h>
static const struct vm_operations_struct xfs_file_vm_ops;
+static const struct vm_operations_struct xfs_file_vm_direct_ops;
+
+static bool
+xfs_vma_is_direct(
+ struct vm_area_struct *vma)
+{
+ return vma->vm_ops == &xfs_file_vm_direct_ops;
+}
/*
* Clear the specified ranges to zero through either the pagecache or DAX.
@@ -1009,6 +1019,22 @@ xfs_file_llseek(
}
/*
+ * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
+ * valid. See map_direct_invalidate.
+ */
+static int
+xfs_can_fault_direct(
+ struct vm_area_struct *vma)
+{
+ if (!xfs_vma_is_direct(vma))
+ return 0;
+
+ if (!test_map_direct_valid(vma->vm_private_data))
+ return VM_FAULT_SIGBUS;
+ return 0;
+}
+
+/*
* Locking for serialisation of IO during page faults. This results in a lock
* ordering of:
*
@@ -1024,7 +1050,8 @@ __xfs_filemap_fault(
enum page_entry_size pe_size,
bool write_fault)
{
- struct inode *inode = file_inode(vmf->vma->vm_file);
+ struct vm_area_struct *vma = vmf->vma;
+ struct inode *inode = file_inode(vma->vm_file);
struct xfs_inode *ip = XFS_I(inode);
int ret;
@@ -1032,10 +1059,14 @@ __xfs_filemap_fault(
if (write_fault) {
sb_start_pagefault(inode->i_sb);
- file_update_time(vmf->vma->vm_file);
+ file_update_time(vma->vm_file);
}
xfs_ilock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
+ ret = xfs_can_fault_direct(vma);
+ if (ret)
+ goto out_unlock;
+
if (IS_DAX(inode)) {
ret = dax_iomap_fault(vmf, pe_size, &xfs_iomap_ops);
} else {
@@ -1044,6 +1075,8 @@ __xfs_filemap_fault(
else
ret = filemap_fault(vmf);
}
+
+out_unlock:
xfs_iunlock(XFS_I(inode), XFS_MMAPLOCK_SHARED);
if (write_fault)
@@ -1115,6 +1148,17 @@ xfs_filemap_pfn_mkwrite(
}
+static const struct vm_operations_struct xfs_file_vm_direct_ops = {
+ .fault = xfs_filemap_fault,
+ .huge_fault = xfs_filemap_huge_fault,
+ .map_pages = filemap_map_pages,
+ .page_mkwrite = xfs_filemap_page_mkwrite,
+ .pfn_mkwrite = xfs_filemap_pfn_mkwrite,
+
+ .open = generic_map_direct_open,
+ .close = generic_map_direct_close,
+};
+
static const struct vm_operations_struct xfs_file_vm_ops = {
.fault = xfs_filemap_fault,
.huge_fault = xfs_filemap_huge_fault,
@@ -1135,6 +1179,60 @@ xfs_file_mmap(
return 0;
}
+static int
+xfs_file_mmap_direct(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ int fd)
+{
+ struct inode *inode = file_inode(filp);
+ struct xfs_inode *ip = XFS_I(inode);
+ struct map_direct_state *mds;
+
+ /*
+ * Not permitted to set up MAP_DIRECT mapping over reflinked or
+ * non-DAX extents since reflink may cause block moves /
+ * copy-on-write, and non-DAX is by definition always indirect
+ * through the page cache.
+ */
+ if (xfs_is_reflink_inode(ip))
+ return -EPERM;
+ if (!IS_DAX(inode))
+ return -EPERM;
+
+ mds = map_direct_register(fd, vma);
+ if (IS_ERR(mds))
+ return PTR_ERR(mds);
+
+ file_accessed(filp);
+ vma->vm_ops = &xfs_file_vm_direct_ops;
+ vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
+
+ /*
+ * generic_map_direct_{open,close} expect ->vm_private_data is
+ * set to the result of map_direct_register
+ */
+ vma->vm_private_data = mds;
+ return 0;
+}
+
+#define XFS_MAP_SUPPORTED (LEGACY_MAP_MASK | MAP_DIRECT)
+
+static int
+xfs_file_mmap_validate(
+ struct file *filp,
+ struct vm_area_struct *vma,
+ unsigned long map_flags,
+ int fd)
+{
+ if (map_flags & ~(XFS_MAP_SUPPORTED))
+ return -EOPNOTSUPP;
+
+ if ((map_flags & MAP_DIRECT) == 0)
+ return xfs_file_mmap(filp, vma);
+ return xfs_file_mmap_direct(filp, vma, fd);
+}
+
const struct file_operations xfs_file_operations = {
.llseek = xfs_file_llseek,
.read_iter = xfs_file_read_iter,
@@ -1146,6 +1244,7 @@ const struct file_operations xfs_file_operations = {
.compat_ioctl = xfs_file_compat_ioctl,
#endif
.mmap = xfs_file_mmap,
+ .mmap_validate = xfs_file_mmap_validate,
.open = xfs_file_open,
.release = xfs_file_release,
.fsync = xfs_file_fsync,
diff --git a/include/linux/mman.h b/include/linux/mman.h
index 94b63b4d71ff..fab393a9dda9 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -20,6 +20,9 @@
#ifndef MAP_HUGE_1GB
#define MAP_HUGE_1GB 0
#endif
+#ifndef MAP_DIRECT
+#define MAP_DIRECT 0
+#endif
#ifndef MAP_UNINITIALIZED
#define MAP_UNINITIALIZED 0
#endif
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index 7162cd4cca73..c916f22008e0 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -12,6 +12,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_DIRECT 0x80000 /* leased block map (layout) for DAX */
/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 1:09 ` Dave Chinner
0 siblings, 0 replies; 77+ messages in thread
From: Dave Chinner @ 2017-10-11 1:09 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, linux-xfs, Jan Kara, Arnd Bergmann,
Darrick J. Wong, linux-rdma, linux-api, iommu, Christoph Hellwig,
J. Bruce Fields, linux-mm, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
> }
>
> /*
> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> + * valid. See map_direct_invalidate.
> + */
> +static int
> +xfs_can_fault_direct(
> + struct vm_area_struct *vma)
> +{
> + if (!xfs_vma_is_direct(vma))
> + return 0;
> +
> + if (!test_map_direct_valid(vma->vm_private_data))
> + return VM_FAULT_SIGBUS;
> + return 0;
> +}
Better, but I'm going to be an annoying pedant here: a "can
<something>" check should return a boolean true/false.
Also, it's a bit jarring to see that a non-direct VMA that /can't/
do direct faults returns the same thing as a direct-vma that /can/
do direct faults, so a couple of extra comments for people who will
quickly forget how this code works (i.e. me) will be helpful. Say
something like this:
/*
* MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
* valid. See map_direct_invalidate.
*/
static bool
xfs_vma_has_direct_lease(
struct vm_area_struct *vma)
{
/* Non MAP_DIRECT vmas do not require layout leases */
if (!xfs_vma_is_direct(vma))
return true;
if (!test_map_direct_valid(vma->vm_private_data))
return false;
/* We have a valid lease */
return true;
}
.....
if (!xfs_vma_has_direct_lease(vma)) {
ret = VM_FAULT_SIGBUS;
goto out_unlock;
}
....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 1:09 ` Dave Chinner
0 siblings, 0 replies; 77+ messages in thread
From: Dave Chinner @ 2017-10-11 1:09 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, linux-xfs, Jan Kara, Arnd Bergmann,
Darrick J. Wong, linux-rdma, linux-api, iommu, Christoph Hellwig,
J. Bruce Fields, linux-mm, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
> }
>
> /*
> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> + * valid. See map_direct_invalidate.
> + */
> +static int
> +xfs_can_fault_direct(
> + struct vm_area_struct *vma)
> +{
> + if (!xfs_vma_is_direct(vma))
> + return 0;
> +
> + if (!test_map_direct_valid(vma->vm_private_data))
> + return VM_FAULT_SIGBUS;
> + return 0;
> +}
Better, but I'm going to be an annoying pedant here: a "can
<something>" check should return a boolean true/false.
Also, it's a bit jarring to see that a non-direct VMA that /can't/
do direct faults returns the same thing as a direct-vma that /can/
do direct faults, so a couple of extra comments for people who will
quickly forget how this code works (i.e. me) will be helpful. Say
something like this:
/*
* MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
* valid. See map_direct_invalidate.
*/
static bool
xfs_vma_has_direct_lease(
struct vm_area_struct *vma)
{
/* Non MAP_DIRECT vmas do not require layout leases */
if (!xfs_vma_is_direct(vma))
return true;
if (!test_map_direct_valid(vma->vm_private_data))
return false;
/* We have a valid lease */
return true;
}
.....
if (!xfs_vma_has_direct_lease(vma)) {
ret = VM_FAULT_SIGBUS;
goto out_unlock;
}
....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 1:09 ` Dave Chinner
0 siblings, 0 replies; 77+ messages in thread
From: Dave Chinner @ 2017-10-11 1:09 UTC (permalink / raw)
To: Dan Williams
Cc: J. Bruce Fields, Jan Kara, Arnd Bergmann, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Alexander Viro, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
Jeff Layton, Christoph Hellwig
On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
> }
>
> /*
> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> + * valid. See map_direct_invalidate.
> + */
> +static int
> +xfs_can_fault_direct(
> + struct vm_area_struct *vma)
> +{
> + if (!xfs_vma_is_direct(vma))
> + return 0;
> +
> + if (!test_map_direct_valid(vma->vm_private_data))
> + return VM_FAULT_SIGBUS;
> + return 0;
> +}
Better, but I'm going to be an annoying pedant here: a "can
<something>" check should return a boolean true/false.
Also, it's a bit jarring to see that a non-direct VMA that /can't/
do direct faults returns the same thing as a direct-vma that /can/
do direct faults, so a couple of extra comments for people who will
quickly forget how this code works (i.e. me) will be helpful. Say
something like this:
/*
* MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
* valid. See map_direct_invalidate.
*/
static bool
xfs_vma_has_direct_lease(
struct vm_area_struct *vma)
{
/* Non MAP_DIRECT vmas do not require layout leases */
if (!xfs_vma_is_direct(vma))
return true;
if (!test_map_direct_valid(vma->vm_private_data))
return false;
/* We have a valid lease */
return true;
}
.....
if (!xfs_vma_has_direct_lease(vma)) {
ret = VM_FAULT_SIGBUS;
goto out_unlock;
}
....
Cheers,
Dave.
--
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
2017-10-11 1:09 ` Dave Chinner
(?)
(?)
@ 2017-10-11 2:12 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: J. Bruce Fields, Jan Kara, Arnd Bergmann, Darrick J. Wong,
linux-rdma, Linux API, linux-nvdimm, linux-xfs, Linux MM, iommu,
Alexander Viro, linux-fsdevel, Jeff Layton, Christoph Hellwig
On Tue, Oct 10, 2017 at 6:09 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
>> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
>> }
>>
>> /*
>> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
>> + * valid. See map_direct_invalidate.
>> + */
>> +static int
>> +xfs_can_fault_direct(
>> + struct vm_area_struct *vma)
>> +{
>> + if (!xfs_vma_is_direct(vma))
>> + return 0;
>> +
>> + if (!test_map_direct_valid(vma->vm_private_data))
>> + return VM_FAULT_SIGBUS;
>> + return 0;
>> +}
>
> Better, but I'm going to be an annoying pedant here: a "can
> <something>" check should return a boolean true/false.
>
> Also, it's a bit jarring to see that a non-direct VMA that /can't/
> do direct faults returns the same thing as a direct-vma that /can/
> do direct faults, so a couple of extra comments for people who will
> quickly forget how this code works (i.e. me) will be helpful. Say
> something like this:
>
> /*
> * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> * valid. See map_direct_invalidate.
> */
> static bool
> xfs_vma_has_direct_lease(
> struct vm_area_struct *vma)
> {
> /* Non MAP_DIRECT vmas do not require layout leases */
> if (!xfs_vma_is_direct(vma))
> return true;
>
> if (!test_map_direct_valid(vma->vm_private_data))
> return false;
>
> /* We have a valid lease */
> return true;
> }
>
> .....
> if (!xfs_vma_has_direct_lease(vma)) {
> ret = VM_FAULT_SIGBUS;
> goto out_unlock;
> }
> ....
Looks good to me.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 2:12 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-nvdimm, linux-xfs, Jan Kara, Arnd Bergmann,
Darrick J. Wong, linux-rdma, Linux API, iommu, Christoph Hellwig,
J. Bruce Fields, Linux MM, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
On Tue, Oct 10, 2017 at 6:09 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
>> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
>> }
>>
>> /*
>> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
>> + * valid. See map_direct_invalidate.
>> + */
>> +static int
>> +xfs_can_fault_direct(
>> + struct vm_area_struct *vma)
>> +{
>> + if (!xfs_vma_is_direct(vma))
>> + return 0;
>> +
>> + if (!test_map_direct_valid(vma->vm_private_data))
>> + return VM_FAULT_SIGBUS;
>> + return 0;
>> +}
>
> Better, but I'm going to be an annoying pedant here: a "can
> <something>" check should return a boolean true/false.
>
> Also, it's a bit jarring to see that a non-direct VMA that /can't/
> do direct faults returns the same thing as a direct-vma that /can/
> do direct faults, so a couple of extra comments for people who will
> quickly forget how this code works (i.e. me) will be helpful. Say
> something like this:
>
> /*
> * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> * valid. See map_direct_invalidate.
> */
> static bool
> xfs_vma_has_direct_lease(
> struct vm_area_struct *vma)
> {
> /* Non MAP_DIRECT vmas do not require layout leases */
> if (!xfs_vma_is_direct(vma))
> return true;
>
> if (!test_map_direct_valid(vma->vm_private_data))
> return false;
>
> /* We have a valid lease */
> return true;
> }
>
> .....
> if (!xfs_vma_has_direct_lease(vma)) {
> ret = VM_FAULT_SIGBUS;
> goto out_unlock;
> }
> ....
Looks good to me.
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 2:12 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: linux-nvdimm, linux-xfs, Jan Kara, Arnd Bergmann,
Darrick J. Wong, linux-rdma, Linux API, iommu, Christoph Hellwig,
J. Bruce Fields, Linux MM, Jeff Moyer, Alexander Viro,
linux-fsdevel, Jeff Layton, Ross Zwisler
On Tue, Oct 10, 2017 at 6:09 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
>> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
>> }
>>
>> /*
>> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
>> + * valid. See map_direct_invalidate.
>> + */
>> +static int
>> +xfs_can_fault_direct(
>> + struct vm_area_struct *vma)
>> +{
>> + if (!xfs_vma_is_direct(vma))
>> + return 0;
>> +
>> + if (!test_map_direct_valid(vma->vm_private_data))
>> + return VM_FAULT_SIGBUS;
>> + return 0;
>> +}
>
> Better, but I'm going to be an annoying pedant here: a "can
> <something>" check should return a boolean true/false.
>
> Also, it's a bit jarring to see that a non-direct VMA that /can't/
> do direct faults returns the same thing as a direct-vma that /can/
> do direct faults, so a couple of extra comments for people who will
> quickly forget how this code works (i.e. me) will be helpful. Say
> something like this:
>
> /*
> * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> * valid. See map_direct_invalidate.
> */
> static bool
> xfs_vma_has_direct_lease(
> struct vm_area_struct *vma)
> {
> /* Non MAP_DIRECT vmas do not require layout leases */
> if (!xfs_vma_is_direct(vma))
> return true;
>
> if (!test_map_direct_valid(vma->vm_private_data))
> return false;
>
> /* We have a valid lease */
> return true;
> }
>
> .....
> if (!xfs_vma_has_direct_lease(vma)) {
> ret = VM_FAULT_SIGBUS;
> goto out_unlock;
> }
> ....
Looks good to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 06/14] xfs: wire up MAP_DIRECT
@ 2017-10-11 2:12 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 2:12 UTC (permalink / raw)
To: Dave Chinner
Cc: J. Bruce Fields, Jan Kara, Arnd Bergmann, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Linux API,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw,
linux-xfs-u79uwXL29TY76Z2rM5mHXA, Linux MM,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Alexander Viro, linux-fsdevel, Jeff Layton, Christoph Hellwig
On Tue, Oct 10, 2017 at 6:09 PM, Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org> wrote:
> On Tue, Oct 10, 2017 at 07:49:30AM -0700, Dan Williams wrote:
>> @@ -1009,6 +1019,22 @@ xfs_file_llseek(
>> }
>>
>> /*
>> + * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
>> + * valid. See map_direct_invalidate.
>> + */
>> +static int
>> +xfs_can_fault_direct(
>> + struct vm_area_struct *vma)
>> +{
>> + if (!xfs_vma_is_direct(vma))
>> + return 0;
>> +
>> + if (!test_map_direct_valid(vma->vm_private_data))
>> + return VM_FAULT_SIGBUS;
>> + return 0;
>> +}
>
> Better, but I'm going to be an annoying pedant here: a "can
> <something>" check should return a boolean true/false.
>
> Also, it's a bit jarring to see that a non-direct VMA that /can't/
> do direct faults returns the same thing as a direct-vma that /can/
> do direct faults, so a couple of extra comments for people who will
> quickly forget how this code works (i.e. me) will be helpful. Say
> something like this:
>
> /*
> * MAP_DIRECT faults can only be serviced while the FL_LAYOUT lease is
> * valid. See map_direct_invalidate.
> */
> static bool
> xfs_vma_has_direct_lease(
> struct vm_area_struct *vma)
> {
> /* Non MAP_DIRECT vmas do not require layout leases */
> if (!xfs_vma_is_direct(vma))
> return true;
>
> if (!test_map_direct_valid(vma->vm_private_data))
> return false;
>
> /* We have a valid lease */
> return true;
> }
>
> .....
> if (!xfs_vma_has_direct_lease(vma)) {
> ret = VM_FAULT_SIGBUS;
> goto out_unlock;
> }
> ....
Looks good to me.
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 07/14] iommu, dma-mapping: introduce dma_get_iommu_domain()
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:49 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Ashok Raj, Darrick J. Wong, linux-rdma,
Greg Kroah-Hartman, Joerg Roedel, Dave Chinner, linux-xfs,
linux-mm, iommu, linux-api, linux-fsdevel, Marek Szyprowski,
David Woodhouse, Christoph Hellwig, Robin Murphy
Add a dma-mapping api helper to retrieve the generic iommu_domain for a
device. The motivation for this interface is making RDMA transfers to
DAX mappings safe. If the DAX file's block map changes we need to be to
reliably stop accesses to blocks that have been freed or re-assigned to
a new file. With the iommu_domain and a callback from the DAX filesystem
the kernel can safely revoke access to a DMA device. The process that
performed the RDMA memory registration is also notified of this
revocation event, but the kernel can not otherwise be in the position of
waiting for userspace to quiesce the device.
Since PMEM+DAX is currently only enabled for x86, we only update the x86
iommu drivers.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/dma-mapping.c | 10 ++++++++++
drivers/iommu/amd_iommu.c | 10 ++++++++++
drivers/iommu/intel-iommu.c | 15 +++++++++++++++
include/linux/dma-mapping.h | 3 +++
4 files changed, 38 insertions(+)
diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index e584eddef0a7..fdb9764f95a4 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -369,3 +369,13 @@ void dma_deconfigure(struct device *dev)
of_dma_deconfigure(dev);
acpi_dma_deconfigure(dev);
}
+
+struct iommu_domain *dma_get_iommu_domain(struct device *dev)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ if (ops && ops->get_iommu)
+ return ops->get_iommu(dev);
+ return NULL;
+}
+EXPORT_SYMBOL(dma_get_iommu_domain);
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 51f8215877f5..c8e1a45af182 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2271,6 +2271,15 @@ static struct protection_domain *get_domain(struct device *dev)
return domain;
}
+static struct iommu_domain *amd_dma_get_iommu(struct device *dev)
+{
+ struct protection_domain *domain = get_domain(dev);
+
+ if (IS_ERR(domain))
+ return NULL;
+ return &domain->domain;
+}
+
static void update_device_table(struct protection_domain *domain)
{
struct iommu_dev_data *dev_data;
@@ -2689,6 +2698,7 @@ static const struct dma_map_ops amd_iommu_dma_ops = {
.unmap_sg = unmap_sg,
.dma_supported = amd_iommu_dma_supported,
.mapping_error = amd_iommu_mapping_error,
+ .get_iommu = amd_dma_get_iommu,
};
static int init_reserved_iova_ranges(void)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6784a05dd6b2..f3f4939cebad 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3578,6 +3578,20 @@ static int iommu_no_mapping(struct device *dev)
return 0;
}
+static struct iommu_domain *intel_dma_get_iommu(struct device *dev)
+{
+ struct dmar_domain *domain;
+
+ if (iommu_no_mapping(dev))
+ return NULL;
+
+ domain = get_valid_domain_for_dev(dev);
+ if (!domain)
+ return NULL;
+
+ return &domain->domain;
+}
+
static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr,
size_t size, int dir, u64 dma_mask)
{
@@ -3872,6 +3886,7 @@ const struct dma_map_ops intel_dma_ops = {
.map_page = intel_map_page,
.unmap_page = intel_unmap_page,
.mapping_error = intel_mapping_error,
+ .get_iommu = intel_dma_get_iommu,
#ifdef CONFIG_X86
.dma_supported = x86_dma_supported,
#endif
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 29ce9815da87..aa62df1d0d72 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -128,6 +128,7 @@ struct dma_map_ops {
enum dma_data_direction dir);
int (*mapping_error)(struct device *dev, dma_addr_t dma_addr);
int (*dma_supported)(struct device *dev, u64 mask);
+ struct iommu_domain *(*get_iommu)(struct device *dev);
#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
u64 (*get_required_mask)(struct device *dev);
#endif
@@ -221,6 +222,8 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
}
#endif
+extern struct iommu_domain *dma_get_iommu_domain(struct device *dev);
+
static inline dma_addr_t dma_map_single_attrs(struct device *dev, void *ptr,
size_t size,
enum dma_data_direction dir,
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 07/14] iommu, dma-mapping: introduce dma_get_iommu_domain()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Ashok Raj, Darrick J. Wong, linux-rdma,
Greg Kroah-Hartman, Joerg Roedel, Dave Chinner, iommu, linux-xfs,
linux-mm, Jeff Moyer, linux-api, linux-fsdevel, Ross Zwisler,
David Woodhouse, Robin Murphy, Christoph Hellwig,
Marek Szyprowski
Add a dma-mapping api helper to retrieve the generic iommu_domain for a
device. The motivation for this interface is making RDMA transfers to
DAX mappings safe. If the DAX file's block map changes we need to be to
reliably stop accesses to blocks that have been freed or re-assigned to
a new file. With the iommu_domain and a callback from the DAX filesystem
the kernel can safely revoke access to a DMA device. The process that
performed the RDMA memory registration is also notified of this
revocation event, but the kernel can not otherwise be in the position of
waiting for userspace to quiesce the device.
Since PMEM+DAX is currently only enabled for x86, we only update the x86
iommu drivers.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/dma-mapping.c | 10 ++++++++++
drivers/iommu/amd_iommu.c | 10 ++++++++++
drivers/iommu/intel-iommu.c | 15 +++++++++++++++
include/linux/dma-mapping.h | 3 +++
4 files changed, 38 insertions(+)
diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index e584eddef0a7..fdb9764f95a4 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -369,3 +369,13 @@ void dma_deconfigure(struct device *dev)
of_dma_deconfigure(dev);
acpi_dma_deconfigure(dev);
}
+
+struct iommu_domain *dma_get_iommu_domain(struct device *dev)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ if (ops && ops->get_iommu)
+ return ops->get_iommu(dev);
+ return NULL;
+}
+EXPORT_SYMBOL(dma_get_iommu_domain);
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 51f8215877f5..c8e1a45af182 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2271,6 +2271,15 @@ static struct protection_domain *get_domain(struct device *dev)
return domain;
}
+static struct iommu_domain *amd_dma_get_iommu(struct device *dev)
+{
+ struct protection_domain *domain = get_domain(dev);
+
+ if (IS_ERR(domain))
+ return NULL;
+ return &domain->domain;
+}
+
static void update_device_table(struct protection_domain *domain)
{
struct iommu_dev_data *dev_data;
@@ -2689,6 +2698,7 @@ static const struct dma_map_ops amd_iommu_dma_ops = {
.unmap_sg = unmap_sg,
.dma_supported = amd_iommu_dma_supported,
.mapping_error = amd_iommu_mapping_error,
+ .get_iommu = amd_dma_get_iommu,
};
static int init_reserved_iova_ranges(void)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6784a05dd6b2..f3f4939cebad 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3578,6 +3578,20 @@ static int iommu_no_mapping(struct device *dev)
return 0;
}
+static struct iommu_domain *intel_dma_get_iommu(struct device *dev)
+{
+ struct dmar_domain *domain;
+
+ if (iommu_no_mapping(dev))
+ return NULL;
+
+ domain = get_valid_domain_for_dev(dev);
+ if (!domain)
+ return NULL;
+
+ return &domain->domain;
+}
+
static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr,
size_t size, int dir, u64 dma_mask)
{
@@ -3872,6 +3886,7 @@ const struct dma_map_ops intel_dma_ops = {
.map_page = intel_map_page,
.unmap_page = intel_unmap_page,
.mapping_error = intel_mapping_error,
+ .get_iommu = intel_dma_get_iommu,
#ifdef CONFIG_X86
.dma_supported = x86_dma_supported,
#endif
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 29ce9815da87..aa62df1d0d72 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -128,6 +128,7 @@ struct dma_map_ops {
enum dma_data_direction dir);
int (*mapping_error)(struct device *dev, dma_addr_t dma_addr);
int (*dma_supported)(struct device *dev, u64 mask);
+ struct iommu_domain *(*get_iommu)(struct device *dev);
#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
u64 (*get_required_mask)(struct device *dev);
#endif
@@ -221,6 +222,8 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
}
#endif
+extern struct iommu_domain *dma_get_iommu_domain(struct device *dev);
+
static inline dma_addr_t dma_map_single_attrs(struct device *dev, void *ptr,
size_t size,
enum dma_data_direction dir,
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 07/14] iommu, dma-mapping: introduce dma_get_iommu_domain()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, Ashok Raj, Darrick J. Wong, linux-rdma,
Greg Kroah-Hartman, Joerg Roedel, Dave Chinner, iommu, linux-xfs,
linux-mm, Jeff Moyer, linux-api, linux-fsdevel, Ross Zwisler,
David Woodhouse, Robin Murphy, Christoph Hellwig,
Marek Szyprowski
Add a dma-mapping api helper to retrieve the generic iommu_domain for a
device. The motivation for this interface is making RDMA transfers to
DAX mappings safe. If the DAX file's block map changes we need to be to
reliably stop accesses to blocks that have been freed or re-assigned to
a new file. With the iommu_domain and a callback from the DAX filesystem
the kernel can safely revoke access to a DMA device. The process that
performed the RDMA memory registration is also notified of this
revocation event, but the kernel can not otherwise be in the position of
waiting for userspace to quiesce the device.
Since PMEM+DAX is currently only enabled for x86, we only update the x86
iommu drivers.
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/dma-mapping.c | 10 ++++++++++
drivers/iommu/amd_iommu.c | 10 ++++++++++
drivers/iommu/intel-iommu.c | 15 +++++++++++++++
include/linux/dma-mapping.h | 3 +++
4 files changed, 38 insertions(+)
diff --git a/drivers/base/dma-mapping.c b/drivers/base/dma-mapping.c
index e584eddef0a7..fdb9764f95a4 100644
--- a/drivers/base/dma-mapping.c
+++ b/drivers/base/dma-mapping.c
@@ -369,3 +369,13 @@ void dma_deconfigure(struct device *dev)
of_dma_deconfigure(dev);
acpi_dma_deconfigure(dev);
}
+
+struct iommu_domain *dma_get_iommu_domain(struct device *dev)
+{
+ const struct dma_map_ops *ops = get_dma_ops(dev);
+
+ if (ops && ops->get_iommu)
+ return ops->get_iommu(dev);
+ return NULL;
+}
+EXPORT_SYMBOL(dma_get_iommu_domain);
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index 51f8215877f5..c8e1a45af182 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2271,6 +2271,15 @@ static struct protection_domain *get_domain(struct device *dev)
return domain;
}
+static struct iommu_domain *amd_dma_get_iommu(struct device *dev)
+{
+ struct protection_domain *domain = get_domain(dev);
+
+ if (IS_ERR(domain))
+ return NULL;
+ return &domain->domain;
+}
+
static void update_device_table(struct protection_domain *domain)
{
struct iommu_dev_data *dev_data;
@@ -2689,6 +2698,7 @@ static const struct dma_map_ops amd_iommu_dma_ops = {
.unmap_sg = unmap_sg,
.dma_supported = amd_iommu_dma_supported,
.mapping_error = amd_iommu_mapping_error,
+ .get_iommu = amd_dma_get_iommu,
};
static int init_reserved_iova_ranges(void)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6784a05dd6b2..f3f4939cebad 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3578,6 +3578,20 @@ static int iommu_no_mapping(struct device *dev)
return 0;
}
+static struct iommu_domain *intel_dma_get_iommu(struct device *dev)
+{
+ struct dmar_domain *domain;
+
+ if (iommu_no_mapping(dev))
+ return NULL;
+
+ domain = get_valid_domain_for_dev(dev);
+ if (!domain)
+ return NULL;
+
+ return &domain->domain;
+}
+
static dma_addr_t __intel_map_single(struct device *dev, phys_addr_t paddr,
size_t size, int dir, u64 dma_mask)
{
@@ -3872,6 +3886,7 @@ const struct dma_map_ops intel_dma_ops = {
.map_page = intel_map_page,
.unmap_page = intel_unmap_page,
.mapping_error = intel_mapping_error,
+ .get_iommu = intel_dma_get_iommu,
#ifdef CONFIG_X86
.dma_supported = x86_dma_supported,
#endif
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 29ce9815da87..aa62df1d0d72 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -128,6 +128,7 @@ struct dma_map_ops {
enum dma_data_direction dir);
int (*mapping_error)(struct device *dev, dma_addr_t dma_addr);
int (*dma_supported)(struct device *dev, u64 mask);
+ struct iommu_domain *(*get_iommu)(struct device *dev);
#ifdef ARCH_HAS_DMA_GET_REQUIRED_MASK
u64 (*get_required_mask)(struct device *dev);
#endif
@@ -221,6 +222,8 @@ static inline const struct dma_map_ops *get_dma_ops(struct device *dev)
}
#endif
+extern struct iommu_domain *dma_get_iommu_domain(struct device *dev);
+
static inline dma_addr_t dma_map_single_attrs(struct device *dev, void *ptr,
size_t size,
enum dma_data_direction dir,
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 08/14] fs, mapdirect: introduce ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: J. Bruce Fields, Jan Kara, Darrick J. Wong, linux-rdma,
linux-api, Dave Chinner, linux-xfs, linux-mm, iommu,
linux-fsdevel, Jeff Layton, Christoph Hellwig
Provide a vma operation that registers a lease that is broken by
break_layout(). This is motivated by a need to stop in-progress RDMA
when the block-map of a DAX-file changes. I.e. since DAX gives
direct-access to filesystem blocks we can not allow those blocks to move
or change state while they are under active RDMA. So, if the filesystem
determines it needs to move blocks it can revoke device access before
proceeding.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/mapdirect.c | 144 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 14 ++++
include/linux/mm.h | 8 +++
3 files changed, 166 insertions(+)
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index 9f4dd7395dcd..c6954033fc1a 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -16,6 +16,7 @@
#include <linux/mutex.h>
#include <linux/sched.h>
#include <linux/slab.h>
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/mm.h>
@@ -32,12 +33,25 @@ struct map_direct_state {
struct vm_area_struct *mds_vma;
};
+struct lease_direct_state {
+ void *lds_owner;
+ struct file *lds_file;
+ unsigned long lds_state;
+ void (*lds_break_fn)(void *lds_owner);
+ struct delayed_work lds_work;
+};
+
bool test_map_direct_valid(struct map_direct_state *mds)
{
return test_bit(MAPDIRECT_VALID, &mds->mds_state);
}
EXPORT_SYMBOL_GPL(test_map_direct_valid);
+static bool test_map_direct_broken(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_BREAK, &mds->mds_state);
+}
+
static void put_map_direct(struct map_direct_state *mds)
{
if (!atomic_dec_and_test(&mds->mds_ref))
@@ -168,6 +182,136 @@ static const struct lock_manager_operations map_direct_lm_ops = {
.lm_setup = map_direct_lm_setup,
};
+static void lease_direct_invalidate(struct work_struct *work)
+{
+ struct lease_direct_state *lds;
+ void *owner;
+
+ lds = container_of(work, typeof(*lds), lds_work.work);
+ owner = lds;
+ lds->lds_break_fn(lds->lds_owner);
+ vfs_setlease(lds->lds_file, F_UNLCK, NULL, &owner);
+}
+
+static bool lease_direct_lm_break(struct file_lock *fl)
+{
+ struct lease_direct_state *lds = fl->fl_owner;
+
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &lds->lds_state))
+ schedule_delayed_work(&lds->lds_work, lease_break_time * HZ);
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int lease_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+ return lease_modify(fl, arg, dispose);
+}
+
+static const struct lock_manager_operations lease_direct_lm_ops = {
+ .lm_break = lease_direct_lm_break,
+ .lm_change = lease_direct_lm_change,
+};
+
+static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner)
+{
+ struct file *file = vma->vm_file;
+ struct lease_direct_state *lds;
+ struct lease_direct *ld;
+ struct file_lock *fl;
+ int rc = -ENOMEM;
+ void *owner;
+
+ ld = kzalloc(sizeof(*ld) + sizeof(*lds), GFP_KERNEL);
+ if (!ld)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&ld->list);
+ lds = (struct lease_direct_state *)(ld + 1);
+ owner = lds;
+ ld->lds = lds;
+ lds->lds_break_fn = lds_break_fn;
+ lds->lds_owner = lds_owner;
+ INIT_DELAYED_WORK(&lds->lds_work, lease_direct_invalidate);
+ lds->lds_file = get_file(file);
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &lease_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = lds;
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = lds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return ld;
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ kfree(lds);
+ return ERR_PTR(rc);
+}
+
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner)
+{
+ struct lease_direct *ld;
+
+ ld = map_direct_lease(vma, break_fn, owner);
+
+ if (IS_ERR(ld))
+ return ld;
+
+ /*
+ * We now have an established lease while the base MAP_DIRECT
+ * lease was not broken. So, we know that the "lease holder" will
+ * receive a SIGIO notification when the lease is broken and
+ * take any necessary cleanup actions.
+ */
+ if (!test_map_direct_broken(vma->vm_private_data))
+ return ld;
+
+ map_direct_lease_destroy(ld);
+
+ return ERR_PTR(-ENXIO);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_lease);
+
+void map_direct_lease_destroy(struct lease_direct *ld)
+{
+ struct lease_direct_state *lds = ld->lds;
+ struct file *file = lds->lds_file;
+ void *owner = lds;
+
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&lds->lds_work);
+ fput(file);
+ WARN_ON(!list_empty(&ld->list));
+ kfree(ld);
+}
+EXPORT_SYMBOL_GPL(map_direct_lease_destroy);
+
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
{
struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index 5491aa550e55..e0df6ac5795a 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -13,17 +13,27 @@
#ifndef __MAPDIRECT_H__
#define __MAPDIRECT_H__
#include <linux/err.h>
+#include <linux/list.h>
struct inode;
struct work_struct;
struct vm_area_struct;
struct map_direct_state;
+struct list_direct_state;
+
+struct lease_direct {
+ struct list_head list;
+ struct lease_direct_state *lds;
+};
#if IS_ENABLED(CONFIG_FS_DAX)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*ld_break_fn)(void *), void *ld_owner);
+void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
struct vm_area_struct *vma)
@@ -36,5 +46,9 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
}
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
+#define generic_map_direct_lease NULL
+static inline void map_direct_lease_destroy(struct lease_direct *ld)
+{
+}
#endif
#endif /* __MAPDIRECT_H__ */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0afa19feb755..00d54e120257 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -420,6 +420,14 @@ struct vm_operations_struct {
*/
struct page *(*find_special_page)(struct vm_area_struct *vma,
unsigned long addr);
+ /*
+ * Called by rdma or similar memory registration agent to
+ * subscribe for "break" events that require any ongoing
+ * accesses, that will not be stopped by a unmap_mapping_range,
+ * to quiesce.
+ */
+ struct lease_direct *(*lease_direct)(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner);
};
struct mmu_gather;
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 08/14] fs, mapdirect: introduce ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
Provide a vma operation that registers a lease that is broken by
break_layout(). This is motivated by a need to stop in-progress RDMA
when the block-map of a DAX-file changes. I.e. since DAX gives
direct-access to filesystem blocks we can not allow those blocks to move
or change state while they are under active RDMA. So, if the filesystem
determines it needs to move blocks it can revoke device access before
proceeding.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/mapdirect.c | 144 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 14 ++++
include/linux/mm.h | 8 +++
3 files changed, 166 insertions(+)
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index 9f4dd7395dcd..c6954033fc1a 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -16,6 +16,7 @@
#include <linux/mutex.h>
#include <linux/sched.h>
#include <linux/slab.h>
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/mm.h>
@@ -32,12 +33,25 @@ struct map_direct_state {
struct vm_area_struct *mds_vma;
};
+struct lease_direct_state {
+ void *lds_owner;
+ struct file *lds_file;
+ unsigned long lds_state;
+ void (*lds_break_fn)(void *lds_owner);
+ struct delayed_work lds_work;
+};
+
bool test_map_direct_valid(struct map_direct_state *mds)
{
return test_bit(MAPDIRECT_VALID, &mds->mds_state);
}
EXPORT_SYMBOL_GPL(test_map_direct_valid);
+static bool test_map_direct_broken(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_BREAK, &mds->mds_state);
+}
+
static void put_map_direct(struct map_direct_state *mds)
{
if (!atomic_dec_and_test(&mds->mds_ref))
@@ -168,6 +182,136 @@ static const struct lock_manager_operations map_direct_lm_ops = {
.lm_setup = map_direct_lm_setup,
};
+static void lease_direct_invalidate(struct work_struct *work)
+{
+ struct lease_direct_state *lds;
+ void *owner;
+
+ lds = container_of(work, typeof(*lds), lds_work.work);
+ owner = lds;
+ lds->lds_break_fn(lds->lds_owner);
+ vfs_setlease(lds->lds_file, F_UNLCK, NULL, &owner);
+}
+
+static bool lease_direct_lm_break(struct file_lock *fl)
+{
+ struct lease_direct_state *lds = fl->fl_owner;
+
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &lds->lds_state))
+ schedule_delayed_work(&lds->lds_work, lease_break_time * HZ);
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int lease_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+ return lease_modify(fl, arg, dispose);
+}
+
+static const struct lock_manager_operations lease_direct_lm_ops = {
+ .lm_break = lease_direct_lm_break,
+ .lm_change = lease_direct_lm_change,
+};
+
+static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner)
+{
+ struct file *file = vma->vm_file;
+ struct lease_direct_state *lds;
+ struct lease_direct *ld;
+ struct file_lock *fl;
+ int rc = -ENOMEM;
+ void *owner;
+
+ ld = kzalloc(sizeof(*ld) + sizeof(*lds), GFP_KERNEL);
+ if (!ld)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&ld->list);
+ lds = (struct lease_direct_state *)(ld + 1);
+ owner = lds;
+ ld->lds = lds;
+ lds->lds_break_fn = lds_break_fn;
+ lds->lds_owner = lds_owner;
+ INIT_DELAYED_WORK(&lds->lds_work, lease_direct_invalidate);
+ lds->lds_file = get_file(file);
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &lease_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = lds;
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = lds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return ld;
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ kfree(lds);
+ return ERR_PTR(rc);
+}
+
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner)
+{
+ struct lease_direct *ld;
+
+ ld = map_direct_lease(vma, break_fn, owner);
+
+ if (IS_ERR(ld))
+ return ld;
+
+ /*
+ * We now have an established lease while the base MAP_DIRECT
+ * lease was not broken. So, we know that the "lease holder" will
+ * receive a SIGIO notification when the lease is broken and
+ * take any necessary cleanup actions.
+ */
+ if (!test_map_direct_broken(vma->vm_private_data))
+ return ld;
+
+ map_direct_lease_destroy(ld);
+
+ return ERR_PTR(-ENXIO);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_lease);
+
+void map_direct_lease_destroy(struct lease_direct *ld)
+{
+ struct lease_direct_state *lds = ld->lds;
+ struct file *file = lds->lds_file;
+ void *owner = lds;
+
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&lds->lds_work);
+ fput(file);
+ WARN_ON(!list_empty(&ld->list));
+ kfree(ld);
+}
+EXPORT_SYMBOL_GPL(map_direct_lease_destroy);
+
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
{
struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index 5491aa550e55..e0df6ac5795a 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -13,17 +13,27 @@
#ifndef __MAPDIRECT_H__
#define __MAPDIRECT_H__
#include <linux/err.h>
+#include <linux/list.h>
struct inode;
struct work_struct;
struct vm_area_struct;
struct map_direct_state;
+struct list_direct_state;
+
+struct lease_direct {
+ struct list_head list;
+ struct lease_direct_state *lds;
+};
#if IS_ENABLED(CONFIG_FS_DAX)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*ld_break_fn)(void *), void *ld_owner);
+void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
struct vm_area_struct *vma)
@@ -36,5 +46,9 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
}
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
+#define generic_map_direct_lease NULL
+static inline void map_direct_lease_destroy(struct lease_direct *ld)
+{
+}
#endif
#endif /* __MAPDIRECT_H__ */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0afa19feb755..00d54e120257 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -420,6 +420,14 @@ struct vm_operations_struct {
*/
struct page *(*find_special_page)(struct vm_area_struct *vma,
unsigned long addr);
+ /*
+ * Called by rdma or similar memory registration agent to
+ * subscribe for "break" events that require any ongoing
+ * accesses, that will not be stopped by a unmap_mapping_range,
+ * to quiesce.
+ */
+ struct lease_direct *(*lease_direct)(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner);
};
struct mmu_gather;
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 08/14] fs, mapdirect: introduce ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
Provide a vma operation that registers a lease that is broken by
break_layout(). This is motivated by a need to stop in-progress RDMA
when the block-map of a DAX-file changes. I.e. since DAX gives
direct-access to filesystem blocks we can not allow those blocks to move
or change state while they are under active RDMA. So, if the filesystem
determines it needs to move blocks it can revoke device access before
proceeding.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/mapdirect.c | 144 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 14 ++++
include/linux/mm.h | 8 +++
3 files changed, 166 insertions(+)
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index 9f4dd7395dcd..c6954033fc1a 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -16,6 +16,7 @@
#include <linux/mutex.h>
#include <linux/sched.h>
#include <linux/slab.h>
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/mm.h>
@@ -32,12 +33,25 @@ struct map_direct_state {
struct vm_area_struct *mds_vma;
};
+struct lease_direct_state {
+ void *lds_owner;
+ struct file *lds_file;
+ unsigned long lds_state;
+ void (*lds_break_fn)(void *lds_owner);
+ struct delayed_work lds_work;
+};
+
bool test_map_direct_valid(struct map_direct_state *mds)
{
return test_bit(MAPDIRECT_VALID, &mds->mds_state);
}
EXPORT_SYMBOL_GPL(test_map_direct_valid);
+static bool test_map_direct_broken(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_BREAK, &mds->mds_state);
+}
+
static void put_map_direct(struct map_direct_state *mds)
{
if (!atomic_dec_and_test(&mds->mds_ref))
@@ -168,6 +182,136 @@ static const struct lock_manager_operations map_direct_lm_ops = {
.lm_setup = map_direct_lm_setup,
};
+static void lease_direct_invalidate(struct work_struct *work)
+{
+ struct lease_direct_state *lds;
+ void *owner;
+
+ lds = container_of(work, typeof(*lds), lds_work.work);
+ owner = lds;
+ lds->lds_break_fn(lds->lds_owner);
+ vfs_setlease(lds->lds_file, F_UNLCK, NULL, &owner);
+}
+
+static bool lease_direct_lm_break(struct file_lock *fl)
+{
+ struct lease_direct_state *lds = fl->fl_owner;
+
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &lds->lds_state))
+ schedule_delayed_work(&lds->lds_work, lease_break_time * HZ);
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int lease_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+ return lease_modify(fl, arg, dispose);
+}
+
+static const struct lock_manager_operations lease_direct_lm_ops = {
+ .lm_break = lease_direct_lm_break,
+ .lm_change = lease_direct_lm_change,
+};
+
+static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner)
+{
+ struct file *file = vma->vm_file;
+ struct lease_direct_state *lds;
+ struct lease_direct *ld;
+ struct file_lock *fl;
+ int rc = -ENOMEM;
+ void *owner;
+
+ ld = kzalloc(sizeof(*ld) + sizeof(*lds), GFP_KERNEL);
+ if (!ld)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&ld->list);
+ lds = (struct lease_direct_state *)(ld + 1);
+ owner = lds;
+ ld->lds = lds;
+ lds->lds_break_fn = lds_break_fn;
+ lds->lds_owner = lds_owner;
+ INIT_DELAYED_WORK(&lds->lds_work, lease_direct_invalidate);
+ lds->lds_file = get_file(file);
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &lease_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = lds;
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = lds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return ld;
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ kfree(lds);
+ return ERR_PTR(rc);
+}
+
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner)
+{
+ struct lease_direct *ld;
+
+ ld = map_direct_lease(vma, break_fn, owner);
+
+ if (IS_ERR(ld))
+ return ld;
+
+ /*
+ * We now have an established lease while the base MAP_DIRECT
+ * lease was not broken. So, we know that the "lease holder" will
+ * receive a SIGIO notification when the lease is broken and
+ * take any necessary cleanup actions.
+ */
+ if (!test_map_direct_broken(vma->vm_private_data))
+ return ld;
+
+ map_direct_lease_destroy(ld);
+
+ return ERR_PTR(-ENXIO);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_lease);
+
+void map_direct_lease_destroy(struct lease_direct *ld)
+{
+ struct lease_direct_state *lds = ld->lds;
+ struct file *file = lds->lds_file;
+ void *owner = lds;
+
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&lds->lds_work);
+ fput(file);
+ WARN_ON(!list_empty(&ld->list));
+ kfree(ld);
+}
+EXPORT_SYMBOL_GPL(map_direct_lease_destroy);
+
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
{
struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index 5491aa550e55..e0df6ac5795a 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -13,17 +13,27 @@
#ifndef __MAPDIRECT_H__
#define __MAPDIRECT_H__
#include <linux/err.h>
+#include <linux/list.h>
struct inode;
struct work_struct;
struct vm_area_struct;
struct map_direct_state;
+struct list_direct_state;
+
+struct lease_direct {
+ struct list_head list;
+ struct lease_direct_state *lds;
+};
#if IS_ENABLED(CONFIG_FS_DAX)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*ld_break_fn)(void *), void *ld_owner);
+void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
struct vm_area_struct *vma)
@@ -36,5 +46,9 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
}
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
+#define generic_map_direct_lease NULL
+static inline void map_direct_lease_destroy(struct lease_direct *ld)
+{
+}
#endif
#endif /* __MAPDIRECT_H__ */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0afa19feb755..00d54e120257 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -420,6 +420,14 @@ struct vm_operations_struct {
*/
struct page *(*find_special_page)(struct vm_area_struct *vma,
unsigned long addr);
+ /*
+ * Called by rdma or similar memory registration agent to
+ * subscribe for "break" events that require any ongoing
+ * accesses, that will not be stopped by a unmap_mapping_range,
+ * to quiesce.
+ */
+ struct lease_direct *(*lease_direct)(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner);
};
struct mmu_gather;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 08/14] fs, mapdirect: introduce ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: J. Bruce Fields, Jan Kara, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Dave Chinner,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Jeff Layton,
Christoph Hellwig
Provide a vma operation that registers a lease that is broken by
break_layout(). This is motivated by a need to stop in-progress RDMA
when the block-map of a DAX-file changes. I.e. since DAX gives
direct-access to filesystem blocks we can not allow those blocks to move
or change state while they are under active RDMA. So, if the filesystem
determines it needs to move blocks it can revoke device access before
proceeding.
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Cc: Jeff Moyer <jmoyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Cc: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Cc: "Darrick J. Wong" <darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: Jeff Layton <jlayton-vpEMnDpepFuMZCB2o+C8xQ@public.gmane.org>
Cc: "J. Bruce Fields" <bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
fs/mapdirect.c | 144 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/mapdirect.h | 14 ++++
include/linux/mm.h | 8 +++
3 files changed, 166 insertions(+)
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index 9f4dd7395dcd..c6954033fc1a 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -16,6 +16,7 @@
#include <linux/mutex.h>
#include <linux/sched.h>
#include <linux/slab.h>
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/mm.h>
@@ -32,12 +33,25 @@ struct map_direct_state {
struct vm_area_struct *mds_vma;
};
+struct lease_direct_state {
+ void *lds_owner;
+ struct file *lds_file;
+ unsigned long lds_state;
+ void (*lds_break_fn)(void *lds_owner);
+ struct delayed_work lds_work;
+};
+
bool test_map_direct_valid(struct map_direct_state *mds)
{
return test_bit(MAPDIRECT_VALID, &mds->mds_state);
}
EXPORT_SYMBOL_GPL(test_map_direct_valid);
+static bool test_map_direct_broken(struct map_direct_state *mds)
+{
+ return test_bit(MAPDIRECT_BREAK, &mds->mds_state);
+}
+
static void put_map_direct(struct map_direct_state *mds)
{
if (!atomic_dec_and_test(&mds->mds_ref))
@@ -168,6 +182,136 @@ static const struct lock_manager_operations map_direct_lm_ops = {
.lm_setup = map_direct_lm_setup,
};
+static void lease_direct_invalidate(struct work_struct *work)
+{
+ struct lease_direct_state *lds;
+ void *owner;
+
+ lds = container_of(work, typeof(*lds), lds_work.work);
+ owner = lds;
+ lds->lds_break_fn(lds->lds_owner);
+ vfs_setlease(lds->lds_file, F_UNLCK, NULL, &owner);
+}
+
+static bool lease_direct_lm_break(struct file_lock *fl)
+{
+ struct lease_direct_state *lds = fl->fl_owner;
+
+ if (!test_and_set_bit(MAPDIRECT_BREAK, &lds->lds_state))
+ schedule_delayed_work(&lds->lds_work, lease_break_time * HZ);
+
+ /* Tell the core lease code to wait for delayed work completion */
+ fl->fl_break_time = 0;
+
+ return false;
+}
+
+static int lease_direct_lm_change(struct file_lock *fl, int arg,
+ struct list_head *dispose)
+{
+ WARN_ON(!(arg & F_UNLCK));
+ return lease_modify(fl, arg, dispose);
+}
+
+static const struct lock_manager_operations lease_direct_lm_ops = {
+ .lm_break = lease_direct_lm_break,
+ .lm_change = lease_direct_lm_change,
+};
+
+static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner)
+{
+ struct file *file = vma->vm_file;
+ struct lease_direct_state *lds;
+ struct lease_direct *ld;
+ struct file_lock *fl;
+ int rc = -ENOMEM;
+ void *owner;
+
+ ld = kzalloc(sizeof(*ld) + sizeof(*lds), GFP_KERNEL);
+ if (!ld)
+ return ERR_PTR(-ENOMEM);
+ INIT_LIST_HEAD(&ld->list);
+ lds = (struct lease_direct_state *)(ld + 1);
+ owner = lds;
+ ld->lds = lds;
+ lds->lds_break_fn = lds_break_fn;
+ lds->lds_owner = lds_owner;
+ INIT_DELAYED_WORK(&lds->lds_work, lease_direct_invalidate);
+ lds->lds_file = get_file(file);
+
+ fl = locks_alloc_lock();
+ if (!fl)
+ goto err_lock_alloc;
+
+ locks_init_lock(fl);
+ fl->fl_lmops = &lease_direct_lm_ops;
+ fl->fl_flags = FL_LAYOUT;
+ fl->fl_type = F_RDLCK;
+ fl->fl_end = OFFSET_MAX;
+ fl->fl_owner = lds;
+ fl->fl_pid = current->tgid;
+ fl->fl_file = file;
+
+ rc = vfs_setlease(file, fl->fl_type, &fl, &owner);
+ if (rc)
+ goto err_setlease;
+ if (fl) {
+ WARN_ON(1);
+ owner = lds;
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ owner = NULL;
+ rc = -ENXIO;
+ goto err_setlease;
+ }
+
+ return ld;
+err_setlease:
+ locks_free_lock(fl);
+err_lock_alloc:
+ kfree(lds);
+ return ERR_PTR(rc);
+}
+
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner)
+{
+ struct lease_direct *ld;
+
+ ld = map_direct_lease(vma, break_fn, owner);
+
+ if (IS_ERR(ld))
+ return ld;
+
+ /*
+ * We now have an established lease while the base MAP_DIRECT
+ * lease was not broken. So, we know that the "lease holder" will
+ * receive a SIGIO notification when the lease is broken and
+ * take any necessary cleanup actions.
+ */
+ if (!test_map_direct_broken(vma->vm_private_data))
+ return ld;
+
+ map_direct_lease_destroy(ld);
+
+ return ERR_PTR(-ENXIO);
+}
+EXPORT_SYMBOL_GPL(generic_map_direct_lease);
+
+void map_direct_lease_destroy(struct lease_direct *ld)
+{
+ struct lease_direct_state *lds = ld->lds;
+ struct file *file = lds->lds_file;
+ void *owner = lds;
+
+ vfs_setlease(file, F_UNLCK, NULL, &owner);
+ flush_delayed_work(&lds->lds_work);
+ fput(file);
+ WARN_ON(!list_empty(&ld->list));
+ kfree(ld);
+}
+EXPORT_SYMBOL_GPL(map_direct_lease_destroy);
+
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma)
{
struct map_direct_state *mds = kzalloc(sizeof(*mds), GFP_KERNEL);
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index 5491aa550e55..e0df6ac5795a 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -13,17 +13,27 @@
#ifndef __MAPDIRECT_H__
#define __MAPDIRECT_H__
#include <linux/err.h>
+#include <linux/list.h>
struct inode;
struct work_struct;
struct vm_area_struct;
struct map_direct_state;
+struct list_direct_state;
+
+struct lease_direct {
+ struct list_head list;
+ struct lease_direct_state *lds;
+};
#if IS_ENABLED(CONFIG_FS_DAX)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
+struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
+ void (*ld_break_fn)(void *), void *ld_owner);
+void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
struct vm_area_struct *vma)
@@ -36,5 +46,9 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
}
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
+#define generic_map_direct_lease NULL
+static inline void map_direct_lease_destroy(struct lease_direct *ld)
+{
+}
#endif
#endif /* __MAPDIRECT_H__ */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0afa19feb755..00d54e120257 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -420,6 +420,14 @@ struct vm_operations_struct {
*/
struct page *(*find_special_page)(struct vm_area_struct *vma,
unsigned long addr);
+ /*
+ * Called by rdma or similar memory registration agent to
+ * subscribe for "break" events that require any ongoing
+ * accesses, that will not be stopped by a unmap_mapping_range,
+ * to quiesce.
+ */
+ struct lease_direct *(*lease_direct)(struct vm_area_struct *vma,
+ void (*break_fn)(void *), void *owner);
};
struct mmu_gather;
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 09/14] xfs: wire up ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: J. Bruce Fields, Jan Kara, Darrick J. Wong, linux-rdma,
linux-api, Dave Chinner, linux-xfs, linux-mm, iommu,
linux-fsdevel, Jeff Layton, Christoph Hellwig
A 'lease_direct' lease requires that the vma have a valid MAP_DIRECT
mapping established. For xfs we use the generic_map_direct_lease()
handler for ->lease_direct(). It establishes a new lease and then checks
if the MAP_DIRECT mapping has been broken. We want to be sure that the
process will receive notification that the MAP_DIRECT mapping is being
torn down so it knows why other code paths are throwing failures.
For example in the RDMA/ibverbs case we want ibv_reg_mr() to fail if the
MAP_DIRECT mapping is invalid or in the process of being invalidated.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_file.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 4bee027c9366..bc512a9a8df5 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1157,6 +1157,7 @@ static const struct vm_operations_struct xfs_file_vm_direct_ops = {
.open = generic_map_direct_open,
.close = generic_map_direct_close,
+ .lease_direct = generic_map_direct_lease,
};
static const struct vm_operations_struct xfs_file_vm_ops = {
@@ -1209,8 +1210,8 @@ xfs_file_mmap_direct(
vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
/*
- * generic_map_direct_{open,close} expect ->vm_private_data is
- * set to the result of map_direct_register
+ * generic_map_direct_{open,close,lease} expect
+ * ->vm_private_data is set to the result of map_direct_register
*/
vma->vm_private_data = mds;
return 0;
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 09/14] xfs: wire up ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
A 'lease_direct' lease requires that the vma have a valid MAP_DIRECT
mapping established. For xfs we use the generic_map_direct_lease()
handler for ->lease_direct(). It establishes a new lease and then checks
if the MAP_DIRECT mapping has been broken. We want to be sure that the
process will receive notification that the MAP_DIRECT mapping is being
torn down so it knows why other code paths are throwing failures.
For example in the RDMA/ibverbs case we want ibv_reg_mr() to fail if the
MAP_DIRECT mapping is invalid or in the process of being invalidated.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_file.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 4bee027c9366..bc512a9a8df5 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1157,6 +1157,7 @@ static const struct vm_operations_struct xfs_file_vm_direct_ops = {
.open = generic_map_direct_open,
.close = generic_map_direct_close,
+ .lease_direct = generic_map_direct_lease,
};
static const struct vm_operations_struct xfs_file_vm_ops = {
@@ -1209,8 +1210,8 @@ xfs_file_mmap_direct(
vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
/*
- * generic_map_direct_{open,close} expect ->vm_private_data is
- * set to the result of map_direct_register
+ * generic_map_direct_{open,close,lease} expect
+ * ->vm_private_data is set to the result of map_direct_register
*/
vma->vm_private_data = mds;
return 0;
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 09/14] xfs: wire up ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-xfs, Jan Kara, Darrick J. Wong, linux-rdma, linux-api,
Dave Chinner, iommu, Christoph Hellwig, J. Bruce Fields,
linux-mm, Jeff Moyer, linux-fsdevel, Jeff Layton, Ross Zwisler
A 'lease_direct' lease requires that the vma have a valid MAP_DIRECT
mapping established. For xfs we use the generic_map_direct_lease()
handler for ->lease_direct(). It establishes a new lease and then checks
if the MAP_DIRECT mapping has been broken. We want to be sure that the
process will receive notification that the MAP_DIRECT mapping is being
torn down so it knows why other code paths are throwing failures.
For example in the RDMA/ibverbs case we want ibv_reg_mr() to fail if the
MAP_DIRECT mapping is invalid or in the process of being invalidated.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
fs/xfs/xfs_file.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 4bee027c9366..bc512a9a8df5 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1157,6 +1157,7 @@ static const struct vm_operations_struct xfs_file_vm_direct_ops = {
.open = generic_map_direct_open,
.close = generic_map_direct_close,
+ .lease_direct = generic_map_direct_lease,
};
static const struct vm_operations_struct xfs_file_vm_ops = {
@@ -1209,8 +1210,8 @@ xfs_file_mmap_direct(
vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
/*
- * generic_map_direct_{open,close} expect ->vm_private_data is
- * set to the result of map_direct_register
+ * generic_map_direct_{open,close,lease} expect
+ * ->vm_private_data is set to the result of map_direct_register
*/
vma->vm_private_data = mds;
return 0;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 09/14] xfs: wire up ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: J. Bruce Fields, Jan Kara, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Dave Chinner,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, Jeff Layton,
Christoph Hellwig
A 'lease_direct' lease requires that the vma have a valid MAP_DIRECT
mapping established. For xfs we use the generic_map_direct_lease()
handler for ->lease_direct(). It establishes a new lease and then checks
if the MAP_DIRECT mapping has been broken. We want to be sure that the
process will receive notification that the MAP_DIRECT mapping is being
torn down so it knows why other code paths are throwing failures.
For example in the RDMA/ibverbs case we want ibv_reg_mr() to fail if the
MAP_DIRECT mapping is invalid or in the process of being invalidated.
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
Cc: Jeff Moyer <jmoyer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Cc: Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>
Cc: "Darrick J. Wong" <darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Cc: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Cc: Jeff Layton <jlayton-vpEMnDpepFuMZCB2o+C8xQ@public.gmane.org>
Cc: "J. Bruce Fields" <bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
fs/xfs/xfs_file.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
index 4bee027c9366..bc512a9a8df5 100644
--- a/fs/xfs/xfs_file.c
+++ b/fs/xfs/xfs_file.c
@@ -1157,6 +1157,7 @@ static const struct vm_operations_struct xfs_file_vm_direct_ops = {
.open = generic_map_direct_open,
.close = generic_map_direct_close,
+ .lease_direct = generic_map_direct_lease,
};
static const struct vm_operations_struct xfs_file_vm_ops = {
@@ -1209,8 +1210,8 @@ xfs_file_mmap_direct(
vma->vm_flags |= VM_MIXEDMAP | VM_HUGEPAGE;
/*
- * generic_map_direct_{open,close} expect ->vm_private_data is
- * set to the result of map_direct_register
+ * generic_map_direct_{open,close,lease} expect
+ * ->vm_private_data is set to the result of map_direct_register
*/
vma->vm_private_data = mds;
return 0;
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 10/14] device-dax: wire up ->lease_direct()
2017-10-10 14:48 ` Dan Williams
@ 2017-10-10 14:49 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, linux-rdma, linux-api, iommu, linux-xfs, linux-mm,
Jeff Moyer, linux-fsdevel, Ross Zwisler, Christoph Hellwig
The only event that will break a lease_direct lease in the device-dax
case is the device shutdown path where the physical pages might get
assigned to another device.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/dax/Kconfig | 1 +
drivers/dax/device.c | 4 ++++
fs/Kconfig | 4 ++++
fs/Makefile | 3 ++-
fs/mapdirect.c | 3 ++-
include/linux/mapdirect.h | 5 ++++-
6 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
index b79aa8f7a497..be03d4dbe646 100644
--- a/drivers/dax/Kconfig
+++ b/drivers/dax/Kconfig
@@ -8,6 +8,7 @@ if DAX
config DEV_DAX
tristate "Device DAX: direct access mapping device"
depends on TRANSPARENT_HUGEPAGE
+ depends on FILE_LOCKING
help
Support raw access to differentiated (persistence, bandwidth,
latency...) memory via an mmap(2) capable character
diff --git a/drivers/dax/device.c b/drivers/dax/device.c
index e9f3b3e4bbf4..fa75004185c4 100644
--- a/drivers/dax/device.c
+++ b/drivers/dax/device.c
@@ -10,6 +10,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
+#include <linux/mapdirect.h>
#include <linux/pagemap.h>
#include <linux/module.h>
#include <linux/device.h>
@@ -430,6 +431,7 @@ static int dev_dax_fault(struct vm_fault *vmf)
static const struct vm_operations_struct dax_vm_ops = {
.fault = dev_dax_fault,
.huge_fault = dev_dax_huge_fault,
+ .lease_direct = map_direct_lease,
};
static int dax_mmap(struct file *filp, struct vm_area_struct *vma)
@@ -540,8 +542,10 @@ static void kill_dev_dax(struct dev_dax *dev_dax)
{
struct dax_device *dax_dev = dev_dax->dax_dev;
struct inode *inode = dax_inode(dax_dev);
+ const bool wait = true;
kill_dax(dax_dev);
+ break_layout(inode, wait);
unmap_mapping_range(inode->i_mapping, 0, 0, 1);
}
diff --git a/fs/Kconfig b/fs/Kconfig
index a7b31a96a753..3668cfb046d5 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -59,6 +59,10 @@ config FS_DAX_PMD
depends on ZONE_DEVICE
depends on TRANSPARENT_HUGEPAGE
+config DAX_MAP_DIRECT
+ bool
+ default FS_DAX || DEV_DAX
+
endif # BLOCK
# Posix ACL utility routines
diff --git a/fs/Makefile b/fs/Makefile
index c0e791d235d8..21b8fb104656 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -29,7 +29,8 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD) += eventfd.o
obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
obj-$(CONFIG_AIO) += aio.o
-obj-$(CONFIG_FS_DAX) += dax.o mapdirect.o
+obj-$(CONFIG_FS_DAX) += dax.o
+obj-$(CONFIG_DAX_MAP_DIRECT) += mapdirect.o
obj-$(CONFIG_FS_ENCRYPTION) += crypto/
obj-$(CONFIG_FILE_LOCKING) += locks.o
obj-$(CONFIG_COMPAT) += compat.o compat_ioctl.o
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index c6954033fc1a..dd4a16f9ffc6 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -218,7 +218,7 @@ static const struct lock_manager_operations lease_direct_lm_ops = {
.lm_change = lease_direct_lm_change,
};
-static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
void (*lds_break_fn)(void *), void *lds_owner)
{
struct file *file = vma->vm_file;
@@ -272,6 +272,7 @@ static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
kfree(lds);
return ERR_PTR(rc);
}
+EXPORT_SYMBOL_GPL(map_direct_lease);
struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
void (*break_fn)(void *), void *owner)
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index e0df6ac5795a..6695fdcf8009 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -26,13 +26,15 @@ struct lease_direct {
struct lease_direct_state *lds;
};
-#if IS_ENABLED(CONFIG_FS_DAX)
+#if IS_ENABLED(CONFIG_DAX_MAP_DIRECT)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
void (*ld_break_fn)(void *), void *ld_owner);
+struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner);
void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
@@ -47,6 +49,7 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
#define generic_map_direct_lease NULL
+#define map_direct_lease NULL
static inline void map_direct_lease_destroy(struct lease_direct *ld)
{
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 10/14] device-dax: wire up ->lease_direct()
@ 2017-10-10 14:49 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:49 UTC (permalink / raw)
To: linux-nvdimm
Cc: Jan Kara, linux-rdma, linux-api, iommu, linux-xfs, linux-mm,
Jeff Moyer, linux-fsdevel, Ross Zwisler, Christoph Hellwig
The only event that will break a lease_direct lease in the device-dax
case is the device shutdown path where the physical pages might get
assigned to another device.
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/dax/Kconfig | 1 +
drivers/dax/device.c | 4 ++++
fs/Kconfig | 4 ++++
fs/Makefile | 3 ++-
fs/mapdirect.c | 3 ++-
include/linux/mapdirect.h | 5 ++++-
6 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
index b79aa8f7a497..be03d4dbe646 100644
--- a/drivers/dax/Kconfig
+++ b/drivers/dax/Kconfig
@@ -8,6 +8,7 @@ if DAX
config DEV_DAX
tristate "Device DAX: direct access mapping device"
depends on TRANSPARENT_HUGEPAGE
+ depends on FILE_LOCKING
help
Support raw access to differentiated (persistence, bandwidth,
latency...) memory via an mmap(2) capable character
diff --git a/drivers/dax/device.c b/drivers/dax/device.c
index e9f3b3e4bbf4..fa75004185c4 100644
--- a/drivers/dax/device.c
+++ b/drivers/dax/device.c
@@ -10,6 +10,7 @@
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*/
+#include <linux/mapdirect.h>
#include <linux/pagemap.h>
#include <linux/module.h>
#include <linux/device.h>
@@ -430,6 +431,7 @@ static int dev_dax_fault(struct vm_fault *vmf)
static const struct vm_operations_struct dax_vm_ops = {
.fault = dev_dax_fault,
.huge_fault = dev_dax_huge_fault,
+ .lease_direct = map_direct_lease,
};
static int dax_mmap(struct file *filp, struct vm_area_struct *vma)
@@ -540,8 +542,10 @@ static void kill_dev_dax(struct dev_dax *dev_dax)
{
struct dax_device *dax_dev = dev_dax->dax_dev;
struct inode *inode = dax_inode(dax_dev);
+ const bool wait = true;
kill_dax(dax_dev);
+ break_layout(inode, wait);
unmap_mapping_range(inode->i_mapping, 0, 0, 1);
}
diff --git a/fs/Kconfig b/fs/Kconfig
index a7b31a96a753..3668cfb046d5 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -59,6 +59,10 @@ config FS_DAX_PMD
depends on ZONE_DEVICE
depends on TRANSPARENT_HUGEPAGE
+config DAX_MAP_DIRECT
+ bool
+ default FS_DAX || DEV_DAX
+
endif # BLOCK
# Posix ACL utility routines
diff --git a/fs/Makefile b/fs/Makefile
index c0e791d235d8..21b8fb104656 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -29,7 +29,8 @@ obj-$(CONFIG_TIMERFD) += timerfd.o
obj-$(CONFIG_EVENTFD) += eventfd.o
obj-$(CONFIG_USERFAULTFD) += userfaultfd.o
obj-$(CONFIG_AIO) += aio.o
-obj-$(CONFIG_FS_DAX) += dax.o mapdirect.o
+obj-$(CONFIG_FS_DAX) += dax.o
+obj-$(CONFIG_DAX_MAP_DIRECT) += mapdirect.o
obj-$(CONFIG_FS_ENCRYPTION) += crypto/
obj-$(CONFIG_FILE_LOCKING) += locks.o
obj-$(CONFIG_COMPAT) += compat.o compat_ioctl.o
diff --git a/fs/mapdirect.c b/fs/mapdirect.c
index c6954033fc1a..dd4a16f9ffc6 100644
--- a/fs/mapdirect.c
+++ b/fs/mapdirect.c
@@ -218,7 +218,7 @@ static const struct lock_manager_operations lease_direct_lm_ops = {
.lm_change = lease_direct_lm_change,
};
-static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
void (*lds_break_fn)(void *), void *lds_owner)
{
struct file *file = vma->vm_file;
@@ -272,6 +272,7 @@ static struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
kfree(lds);
return ERR_PTR(rc);
}
+EXPORT_SYMBOL_GPL(map_direct_lease);
struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
void (*break_fn)(void *), void *owner)
diff --git a/include/linux/mapdirect.h b/include/linux/mapdirect.h
index e0df6ac5795a..6695fdcf8009 100644
--- a/include/linux/mapdirect.h
+++ b/include/linux/mapdirect.h
@@ -26,13 +26,15 @@ struct lease_direct {
struct lease_direct_state *lds;
};
-#if IS_ENABLED(CONFIG_FS_DAX)
+#if IS_ENABLED(CONFIG_DAX_MAP_DIRECT)
struct map_direct_state *map_direct_register(int fd, struct vm_area_struct *vma);
bool test_map_direct_valid(struct map_direct_state *mds);
void generic_map_direct_open(struct vm_area_struct *vma);
void generic_map_direct_close(struct vm_area_struct *vma);
struct lease_direct *generic_map_direct_lease(struct vm_area_struct *vma,
void (*ld_break_fn)(void *), void *ld_owner);
+struct lease_direct *map_direct_lease(struct vm_area_struct *vma,
+ void (*lds_break_fn)(void *), void *lds_owner);
void map_direct_lease_destroy(struct lease_direct *ld);
#else
static inline struct map_direct_state *map_direct_register(int fd,
@@ -47,6 +49,7 @@ static inline bool test_map_direct_valid(struct map_direct_state *mds)
#define generic_map_direct_open NULL
#define generic_map_direct_close NULL
#define generic_map_direct_lease NULL
+#define map_direct_lease NULL
static inline void map_direct_lease_destroy(struct lease_direct *ld)
{
}
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 11/14] iommu: up-level sg_num_pages() from amd-iommu
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:50 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, Joerg Roedel, linux-xfs, linux-mm, iommu,
linux-fsdevel
iommu_sg_num_pages() is a helper that walks a scattlerlist and counts
pages taking segment boundaries and iommu_num_pages() into account.
Up-level it for determining the IOVA range that dma_map_ops established
at dma_map_sg() time. The intent is to iommu_unmap() the IOVA range in
advance of freeing IOVA range.
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/amd_iommu.c | 30 ++----------------------------
drivers/iommu/iommu.c | 27 +++++++++++++++++++++++++++
include/linux/iommu.h | 2 ++
3 files changed, 31 insertions(+), 28 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index c8e1a45af182..4795b0823469 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2459,32 +2459,6 @@ static void unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
__unmap_single(dma_dom, dma_addr, size, dir);
}
-static int sg_num_pages(struct device *dev,
- struct scatterlist *sglist,
- int nelems)
-{
- unsigned long mask, boundary_size;
- struct scatterlist *s;
- int i, npages = 0;
-
- mask = dma_get_seg_boundary(dev);
- boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT :
- 1UL << (BITS_PER_LONG - PAGE_SHIFT);
-
- for_each_sg(sglist, s, nelems, i) {
- int p, n;
-
- s->dma_address = npages << PAGE_SHIFT;
- p = npages % boundary_size;
- n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
- if (p + n > boundary_size)
- npages += boundary_size - p;
- npages += n;
- }
-
- return npages;
-}
-
/*
* The exported map_sg function for dma_ops (handles scatter-gather
* lists).
@@ -2507,7 +2481,7 @@ static int map_sg(struct device *dev, struct scatterlist *sglist,
dma_dom = to_dma_ops_domain(domain);
dma_mask = *dev->dma_mask;
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
address = dma_ops_alloc_iova(dev, dma_dom, npages, dma_mask);
if (address == AMD_IOMMU_MAPPING_ERROR)
@@ -2585,7 +2559,7 @@ static void unmap_sg(struct device *dev, struct scatterlist *sglist,
startaddr = sg_dma_address(sglist) & PAGE_MASK;
dma_dom = to_dma_ops_domain(domain);
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
__unmap_single(dma_dom, startaddr, npages << PAGE_SHIFT, dir);
}
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 3de5c0bcb5cc..cfe6eeea3578 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -33,6 +33,7 @@
#include <linux/bitops.h>
#include <linux/property.h>
#include <trace/events/iommu.h>
+#include <linux/iommu-helper.h>
static struct kset *iommu_group_kset;
static DEFINE_IDA(iommu_group_ida);
@@ -1631,6 +1632,32 @@ size_t iommu_unmap_fast(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_unmap_fast);
+int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems)
+{
+ unsigned long mask, boundary_size;
+ struct scatterlist *s;
+ int i, npages = 0;
+
+ mask = dma_get_seg_boundary(dev);
+ boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT
+ : 1UL << (BITS_PER_LONG - PAGE_SHIFT);
+
+ for_each_sg(sglist, s, nelems, i) {
+ int p, n;
+
+ s->dma_address = npages << PAGE_SHIFT;
+ p = npages % boundary_size;
+ n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
+ if (p + n > boundary_size)
+ npages += boundary_size - p;
+ npages += n;
+ }
+
+ return npages;
+}
+EXPORT_SYMBOL_GPL(iommu_sg_num_pages);
+
size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg, unsigned int nents, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index a7f2ac689d29..5b2d20e1475a 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -303,6 +303,8 @@ extern size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
size_t size);
extern size_t iommu_unmap_fast(struct iommu_domain *domain,
unsigned long iova, size_t size);
+extern int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems);
extern size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg,unsigned int nents,
int prot);
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 11/14] iommu: up-level sg_num_pages() from amd-iommu
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, Joerg Roedel, linux-xfs, linux-mm, iommu,
linux-fsdevel
iommu_sg_num_pages() is a helper that walks a scattlerlist and counts
pages taking segment boundaries and iommu_num_pages() into account.
Up-level it for determining the IOVA range that dma_map_ops established
at dma_map_sg() time. The intent is to iommu_unmap() the IOVA range in
advance of freeing IOVA range.
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/amd_iommu.c | 30 ++----------------------------
drivers/iommu/iommu.c | 27 +++++++++++++++++++++++++++
include/linux/iommu.h | 2 ++
3 files changed, 31 insertions(+), 28 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index c8e1a45af182..4795b0823469 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2459,32 +2459,6 @@ static void unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
__unmap_single(dma_dom, dma_addr, size, dir);
}
-static int sg_num_pages(struct device *dev,
- struct scatterlist *sglist,
- int nelems)
-{
- unsigned long mask, boundary_size;
- struct scatterlist *s;
- int i, npages = 0;
-
- mask = dma_get_seg_boundary(dev);
- boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT :
- 1UL << (BITS_PER_LONG - PAGE_SHIFT);
-
- for_each_sg(sglist, s, nelems, i) {
- int p, n;
-
- s->dma_address = npages << PAGE_SHIFT;
- p = npages % boundary_size;
- n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
- if (p + n > boundary_size)
- npages += boundary_size - p;
- npages += n;
- }
-
- return npages;
-}
-
/*
* The exported map_sg function for dma_ops (handles scatter-gather
* lists).
@@ -2507,7 +2481,7 @@ static int map_sg(struct device *dev, struct scatterlist *sglist,
dma_dom = to_dma_ops_domain(domain);
dma_mask = *dev->dma_mask;
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
address = dma_ops_alloc_iova(dev, dma_dom, npages, dma_mask);
if (address == AMD_IOMMU_MAPPING_ERROR)
@@ -2585,7 +2559,7 @@ static void unmap_sg(struct device *dev, struct scatterlist *sglist,
startaddr = sg_dma_address(sglist) & PAGE_MASK;
dma_dom = to_dma_ops_domain(domain);
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
__unmap_single(dma_dom, startaddr, npages << PAGE_SHIFT, dir);
}
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 3de5c0bcb5cc..cfe6eeea3578 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -33,6 +33,7 @@
#include <linux/bitops.h>
#include <linux/property.h>
#include <trace/events/iommu.h>
+#include <linux/iommu-helper.h>
static struct kset *iommu_group_kset;
static DEFINE_IDA(iommu_group_ida);
@@ -1631,6 +1632,32 @@ size_t iommu_unmap_fast(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_unmap_fast);
+int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems)
+{
+ unsigned long mask, boundary_size;
+ struct scatterlist *s;
+ int i, npages = 0;
+
+ mask = dma_get_seg_boundary(dev);
+ boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT
+ : 1UL << (BITS_PER_LONG - PAGE_SHIFT);
+
+ for_each_sg(sglist, s, nelems, i) {
+ int p, n;
+
+ s->dma_address = npages << PAGE_SHIFT;
+ p = npages % boundary_size;
+ n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
+ if (p + n > boundary_size)
+ npages += boundary_size - p;
+ npages += n;
+ }
+
+ return npages;
+}
+EXPORT_SYMBOL_GPL(iommu_sg_num_pages);
+
size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg, unsigned int nents, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index a7f2ac689d29..5b2d20e1475a 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -303,6 +303,8 @@ extern size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
size_t size);
extern size_t iommu_unmap_fast(struct iommu_domain *domain,
unsigned long iova, size_t size);
+extern int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems);
extern size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg,unsigned int nents,
int prot);
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 11/14] iommu: up-level sg_num_pages() from amd-iommu
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, Joerg Roedel, linux-xfs, linux-mm, iommu,
linux-fsdevel
iommu_sg_num_pages() is a helper that walks a scattlerlist and counts
pages taking segment boundaries and iommu_num_pages() into account.
Up-level it for determining the IOVA range that dma_map_ops established
at dma_map_sg() time. The intent is to iommu_unmap() the IOVA range in
advance of freeing IOVA range.
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/amd_iommu.c | 30 ++----------------------------
drivers/iommu/iommu.c | 27 +++++++++++++++++++++++++++
include/linux/iommu.h | 2 ++
3 files changed, 31 insertions(+), 28 deletions(-)
diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
index c8e1a45af182..4795b0823469 100644
--- a/drivers/iommu/amd_iommu.c
+++ b/drivers/iommu/amd_iommu.c
@@ -2459,32 +2459,6 @@ static void unmap_page(struct device *dev, dma_addr_t dma_addr, size_t size,
__unmap_single(dma_dom, dma_addr, size, dir);
}
-static int sg_num_pages(struct device *dev,
- struct scatterlist *sglist,
- int nelems)
-{
- unsigned long mask, boundary_size;
- struct scatterlist *s;
- int i, npages = 0;
-
- mask = dma_get_seg_boundary(dev);
- boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT :
- 1UL << (BITS_PER_LONG - PAGE_SHIFT);
-
- for_each_sg(sglist, s, nelems, i) {
- int p, n;
-
- s->dma_address = npages << PAGE_SHIFT;
- p = npages % boundary_size;
- n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
- if (p + n > boundary_size)
- npages += boundary_size - p;
- npages += n;
- }
-
- return npages;
-}
-
/*
* The exported map_sg function for dma_ops (handles scatter-gather
* lists).
@@ -2507,7 +2481,7 @@ static int map_sg(struct device *dev, struct scatterlist *sglist,
dma_dom = to_dma_ops_domain(domain);
dma_mask = *dev->dma_mask;
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
address = dma_ops_alloc_iova(dev, dma_dom, npages, dma_mask);
if (address == AMD_IOMMU_MAPPING_ERROR)
@@ -2585,7 +2559,7 @@ static void unmap_sg(struct device *dev, struct scatterlist *sglist,
startaddr = sg_dma_address(sglist) & PAGE_MASK;
dma_dom = to_dma_ops_domain(domain);
- npages = sg_num_pages(dev, sglist, nelems);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
__unmap_single(dma_dom, startaddr, npages << PAGE_SHIFT, dir);
}
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 3de5c0bcb5cc..cfe6eeea3578 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -33,6 +33,7 @@
#include <linux/bitops.h>
#include <linux/property.h>
#include <trace/events/iommu.h>
+#include <linux/iommu-helper.h>
static struct kset *iommu_group_kset;
static DEFINE_IDA(iommu_group_ida);
@@ -1631,6 +1632,32 @@ size_t iommu_unmap_fast(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_unmap_fast);
+int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems)
+{
+ unsigned long mask, boundary_size;
+ struct scatterlist *s;
+ int i, npages = 0;
+
+ mask = dma_get_seg_boundary(dev);
+ boundary_size = mask + 1 ? ALIGN(mask + 1, PAGE_SIZE) >> PAGE_SHIFT
+ : 1UL << (BITS_PER_LONG - PAGE_SHIFT);
+
+ for_each_sg(sglist, s, nelems, i) {
+ int p, n;
+
+ s->dma_address = npages << PAGE_SHIFT;
+ p = npages % boundary_size;
+ n = iommu_num_pages(sg_phys(s), s->length, PAGE_SIZE);
+ if (p + n > boundary_size)
+ npages += boundary_size - p;
+ npages += n;
+ }
+
+ return npages;
+}
+EXPORT_SYMBOL_GPL(iommu_sg_num_pages);
+
size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg, unsigned int nents, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index a7f2ac689d29..5b2d20e1475a 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -303,6 +303,8 @@ extern size_t iommu_unmap(struct iommu_domain *domain, unsigned long iova,
size_t size);
extern size_t iommu_unmap_fast(struct iommu_domain *domain,
unsigned long iova, size_t size);
+extern int iommu_sg_num_pages(struct device *dev, struct scatterlist *sglist,
+ int nelems);
extern size_t default_iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
struct scatterlist *sg,unsigned int nents,
int prot);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 12/14] iommu/vt-d: use iommu_num_sg_pages
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: Ashok Raj, linux-rdma, linux-api, Joerg Roedel, linux-xfs,
linux-mm, iommu, linux-fsdevel, David Woodhouse
Use the common helper for accounting the size of the IOVA range for a
scatterlist so that iommu and dma apis agree on the size of a
scatterlist. This is in support for using iommu_unmap() in advance of
dma_unmap_sg() to invalidate an io-mapping in advance of the IOVA range
being deallocated. MAP_DIRECT needs this functionality for force
revoking RDMA access to a DAX mapping when userspace fails to respond to
within a lease break timeout period.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/intel-iommu.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3f4939cebad..94a5fbe62fb8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3785,14 +3785,9 @@ static void intel_unmap_sg(struct device *dev, struct scatterlist *sglist,
unsigned long attrs)
{
dma_addr_t startaddr = sg_dma_address(sglist) & PAGE_MASK;
- unsigned long nrpages = 0;
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sglist, sg, nelems, i) {
- nrpages += aligned_nrpages(sg_dma_address(sg), sg_dma_len(sg));
- }
+ unsigned long nrpages;
+ nrpages = iommu_sg_num_pages(dev, sglist, nelems);
intel_unmap(dev, startaddr, nrpages << VTD_PAGE_SHIFT);
}
@@ -3813,14 +3808,12 @@ static int intel_nontranslate_map_sg(struct device *hddev,
static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nelems,
enum dma_data_direction dir, unsigned long attrs)
{
- int i;
struct dmar_domain *domain;
size_t size = 0;
int prot = 0;
unsigned long iova_pfn;
int ret;
- struct scatterlist *sg;
- unsigned long start_vpfn;
+ unsigned long start_vpfn, npages;
struct intel_iommu *iommu;
BUG_ON(dir == DMA_NONE);
@@ -3833,11 +3826,9 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nele
iommu = domain_get_iommu(domain);
- for_each_sg(sglist, sg, nelems, i)
- size += aligned_nrpages(sg->offset, sg->length);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
- iova_pfn = intel_alloc_iova(dev, domain, dma_to_mm_pfn(size),
- *dev->dma_mask);
+ iova_pfn = intel_alloc_iova(dev, domain, npages, *dev->dma_mask);
if (!iova_pfn) {
sglist->dma_length = 0;
return 0;
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 12/14] iommu/vt-d: use iommu_num_sg_pages
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: Ashok Raj, linux-rdma, linux-api, Joerg Roedel, linux-xfs,
linux-mm, iommu, linux-fsdevel, David Woodhouse
Use the common helper for accounting the size of the IOVA range for a
scatterlist so that iommu and dma apis agree on the size of a
scatterlist. This is in support for using iommu_unmap() in advance of
dma_unmap_sg() to invalidate an io-mapping in advance of the IOVA range
being deallocated. MAP_DIRECT needs this functionality for force
revoking RDMA access to a DAX mapping when userspace fails to respond to
within a lease break timeout period.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/intel-iommu.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3f4939cebad..94a5fbe62fb8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3785,14 +3785,9 @@ static void intel_unmap_sg(struct device *dev, struct scatterlist *sglist,
unsigned long attrs)
{
dma_addr_t startaddr = sg_dma_address(sglist) & PAGE_MASK;
- unsigned long nrpages = 0;
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sglist, sg, nelems, i) {
- nrpages += aligned_nrpages(sg_dma_address(sg), sg_dma_len(sg));
- }
+ unsigned long nrpages;
+ nrpages = iommu_sg_num_pages(dev, sglist, nelems);
intel_unmap(dev, startaddr, nrpages << VTD_PAGE_SHIFT);
}
@@ -3813,14 +3808,12 @@ static int intel_nontranslate_map_sg(struct device *hddev,
static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nelems,
enum dma_data_direction dir, unsigned long attrs)
{
- int i;
struct dmar_domain *domain;
size_t size = 0;
int prot = 0;
unsigned long iova_pfn;
int ret;
- struct scatterlist *sg;
- unsigned long start_vpfn;
+ unsigned long start_vpfn, npages;
struct intel_iommu *iommu;
BUG_ON(dir == DMA_NONE);
@@ -3833,11 +3826,9 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nele
iommu = domain_get_iommu(domain);
- for_each_sg(sglist, sg, nelems, i)
- size += aligned_nrpages(sg->offset, sg->length);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
- iova_pfn = intel_alloc_iova(dev, domain, dma_to_mm_pfn(size),
- *dev->dma_mask);
+ iova_pfn = intel_alloc_iova(dev, domain, npages, *dev->dma_mask);
if (!iova_pfn) {
sglist->dma_length = 0;
return 0;
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 12/14] iommu/vt-d: use iommu_num_sg_pages
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: Ashok Raj, linux-rdma, linux-api, Joerg Roedel, linux-xfs,
linux-mm, iommu, linux-fsdevel, David Woodhouse
Use the common helper for accounting the size of the IOVA range for a
scatterlist so that iommu and dma apis agree on the size of a
scatterlist. This is in support for using iommu_unmap() in advance of
dma_unmap_sg() to invalidate an io-mapping in advance of the IOVA range
being deallocated. MAP_DIRECT needs this functionality for force
revoking RDMA access to a DAX mapping when userspace fails to respond to
within a lease break timeout period.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/iommu/intel-iommu.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3f4939cebad..94a5fbe62fb8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3785,14 +3785,9 @@ static void intel_unmap_sg(struct device *dev, struct scatterlist *sglist,
unsigned long attrs)
{
dma_addr_t startaddr = sg_dma_address(sglist) & PAGE_MASK;
- unsigned long nrpages = 0;
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sglist, sg, nelems, i) {
- nrpages += aligned_nrpages(sg_dma_address(sg), sg_dma_len(sg));
- }
+ unsigned long nrpages;
+ nrpages = iommu_sg_num_pages(dev, sglist, nelems);
intel_unmap(dev, startaddr, nrpages << VTD_PAGE_SHIFT);
}
@@ -3813,14 +3808,12 @@ static int intel_nontranslate_map_sg(struct device *hddev,
static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nelems,
enum dma_data_direction dir, unsigned long attrs)
{
- int i;
struct dmar_domain *domain;
size_t size = 0;
int prot = 0;
unsigned long iova_pfn;
int ret;
- struct scatterlist *sg;
- unsigned long start_vpfn;
+ unsigned long start_vpfn, npages;
struct intel_iommu *iommu;
BUG_ON(dir == DMA_NONE);
@@ -3833,11 +3826,9 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nele
iommu = domain_get_iommu(domain);
- for_each_sg(sglist, sg, nelems, i)
- size += aligned_nrpages(sg->offset, sg->length);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
- iova_pfn = intel_alloc_iova(dev, domain, dma_to_mm_pfn(size),
- *dev->dma_mask);
+ iova_pfn = intel_alloc_iova(dev, domain, npages, *dev->dma_mask);
if (!iova_pfn) {
sglist->dma_length = 0;
return 0;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 12/14] iommu/vt-d: use iommu_num_sg_pages
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: Ashok Raj, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Joerg Roedel,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA, David Woodhouse
Use the common helper for accounting the size of the IOVA range for a
scatterlist so that iommu and dma apis agree on the size of a
scatterlist. This is in support for using iommu_unmap() in advance of
dma_unmap_sg() to invalidate an io-mapping in advance of the IOVA range
being deallocated. MAP_DIRECT needs this functionality for force
revoking RDMA access to a DAX mapping when userspace fails to respond to
within a lease break timeout period.
Cc: Ashok Raj <ashok.raj-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: David Woodhouse <dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
Cc: Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/iommu/intel-iommu.c | 19 +++++--------------
1 file changed, 5 insertions(+), 14 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f3f4939cebad..94a5fbe62fb8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3785,14 +3785,9 @@ static void intel_unmap_sg(struct device *dev, struct scatterlist *sglist,
unsigned long attrs)
{
dma_addr_t startaddr = sg_dma_address(sglist) & PAGE_MASK;
- unsigned long nrpages = 0;
- struct scatterlist *sg;
- int i;
-
- for_each_sg(sglist, sg, nelems, i) {
- nrpages += aligned_nrpages(sg_dma_address(sg), sg_dma_len(sg));
- }
+ unsigned long nrpages;
+ nrpages = iommu_sg_num_pages(dev, sglist, nelems);
intel_unmap(dev, startaddr, nrpages << VTD_PAGE_SHIFT);
}
@@ -3813,14 +3808,12 @@ static int intel_nontranslate_map_sg(struct device *hddev,
static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nelems,
enum dma_data_direction dir, unsigned long attrs)
{
- int i;
struct dmar_domain *domain;
size_t size = 0;
int prot = 0;
unsigned long iova_pfn;
int ret;
- struct scatterlist *sg;
- unsigned long start_vpfn;
+ unsigned long start_vpfn, npages;
struct intel_iommu *iommu;
BUG_ON(dir == DMA_NONE);
@@ -3833,11 +3826,9 @@ static int intel_map_sg(struct device *dev, struct scatterlist *sglist, int nele
iommu = domain_get_iommu(domain);
- for_each_sg(sglist, sg, nelems, i)
- size += aligned_nrpages(sg->offset, sg->length);
+ npages = iommu_sg_num_pages(dev, sglist, nelems);
- iova_pfn = intel_alloc_iova(dev, domain, dma_to_mm_pfn(size),
- *dev->dma_mask);
+ iova_pfn = intel_alloc_iova(dev, domain, npages, *dev->dma_mask);
if (!iova_pfn) {
sglist->dma_length = 0;
return 0;
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
2017-10-10 14:48 ` Dan Williams
(?)
@ 2017-10-10 14:50 ` Dan Williams
-1 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: J. Bruce Fields, Doug Ledford, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, linux-api, Joerg Roedel,
Dave Chinner, iommu, Hal Rosenstock, linux-xfs, linux-mm,
Jeff Layton, linux-fsdevel, Sean Hefty, David Woodhouse,
Christoph Hellwig
Currently the ibverbs core in the kernel is completely unaware of the
dangers of filesystem-DAX mappings. Specifically, the filesystem is free
to move file blocks at will. In the case of DAX, it means that RDMA to a
given file offset can dynamically switch to another file offset, another
file, or free space with no notification to RDMA device to cease
operations. Historically, this lack of communication between the ibverbs
core and filesystem was not a problem because RDMA always targeted
dynamically allocated page cache, so at least the RDMA device would have
valid memory to target even if the file was being modified. With DAX we
need to add coordination since RDMA is bypassing page-cache and going
direct to on-media pages of the file. RDMA to DAX can cause damage if
filesystem blocks move / change state.
Use the new ->lease_direct() operation to get a notification when the
filesystem is invalidating the block map of the file and needs RDMA
operations to stop. Given that the kernel can not be in a position where
it needs to wait indefinitely for userspace to stop a device we need a
mechanism where the kernel can force-revoke access. Towards that end, use
the dma_get_iommu_domain() to both check if the device has domain
mappings that can be invalidated and retrieve the iommu_domain for use
with iommu_unmap.
Once we have that assurance that we can block in-flight I/O when the
file's block map changes then we can safely allow RDMA to DAX.
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Doug Ledford <dledford@redhat.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/infiniband/core/umem.c | 90 +++++++++++++++++++++++++++++++++++-----
include/rdma/ib_umem.h | 8 ++++
2 files changed, 86 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 21e60b1e2ff4..5e4598982359 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -36,6 +36,7 @@
#include <linux/dma-mapping.h>
#include <linux/sched/signal.h>
#include <linux/sched/mm.h>
+#include <linux/mapdirect.h>
#include <linux/export.h>
#include <linux/hugetlb.h>
#include <linux/slab.h>
@@ -46,10 +47,16 @@
static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty)
{
+ struct lease_direct *ld, *_ld;
struct scatterlist *sg;
struct page *page;
int i;
+ list_for_each_entry_safe(ld, _ld, &umem->leases, list) {
+ list_del_init(&ld->list);
+ map_direct_lease_destroy(ld);
+ }
+
if (umem->nmap > 0)
ib_dma_unmap_sg(dev, umem->sg_head.sgl,
umem->npages,
@@ -64,10 +71,20 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
}
sg_free_table(&umem->sg_head);
- return;
}
+static void ib_umem_lease_break(void *__umem)
+{
+ struct ib_umem *umem = umem;
+ struct ib_device *idev = umem->context->device;
+ struct device *dev = idev->dma_device;
+ struct scatterlist *sgl = umem->sg_head.sgl;
+
+ iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
+ iommu_sg_num_pages(dev, sgl, umem->npages));
+}
+
/**
* ib_umem_get - Pin and DMA map userspace memory.
*
@@ -96,7 +113,10 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
struct scatterlist *sg, *sg_list_start;
int need_release = 0;
unsigned int gup_flags = FOLL_WRITE;
+ struct vm_area_struct *vma_prev = NULL;
+ struct device *dma_dev;
+ dma_dev = context->device->dma_device;
if (dmasync)
dma_attrs |= DMA_ATTR_WRITE_BARRIER;
@@ -120,6 +140,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->address = addr;
umem->page_shift = PAGE_SHIFT;
umem->pid = get_task_pid(current, PIDTYPE_PID);
+ INIT_LIST_HEAD(&umem->leases);
/*
* We ask for writable memory if any of the following
* access flags are set. "Local write" and "remote write"
@@ -147,19 +168,21 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->hugetlb = 1;
page_list = (struct page **) __get_free_page(GFP_KERNEL);
- if (!page_list) {
- put_pid(umem->pid);
- kfree(umem);
- return ERR_PTR(-ENOMEM);
- }
+ if (!page_list)
+ goto err_pagelist;
/*
- * if we can't alloc the vma_list, it's not so bad;
- * just assume the memory is not hugetlb memory
+ * If DAX is enabled we need the vma to setup a ->lease_direct()
+ * lease to protect against file modifications, otherwise we can
+ * tolerate a failure to allocate the vma_list and just assume
+ * that all vmas are not hugetlb-vmas.
*/
vma_list = (struct vm_area_struct **) __get_free_page(GFP_KERNEL);
- if (!vma_list)
+ if (!vma_list) {
+ if (IS_ENABLED(CONFIG_DAX_MAP_DIRECT))
+ goto err_vmalist;
umem->hugetlb = 0;
+ }
npages = ib_umem_num_pages(umem);
@@ -199,15 +222,52 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
if (ret < 0)
goto out;
- umem->npages += ret;
cur_base += ret * PAGE_SIZE;
npages -= ret;
for_each_sg(sg_list_start, sg, ret, i) {
- if (vma_list && !is_vm_hugetlb_page(vma_list[i]))
- umem->hugetlb = 0;
+ const struct vm_operations_struct *vm_ops;
+ struct vm_area_struct *vma;
+ struct lease_direct *ld;
sg_set_page(sg, page_list[i], PAGE_SIZE, 0);
+ umem->npages++;
+
+ if (!vma_list)
+ continue;
+ vma = vma_list[i];
+
+ if (vma == vma_prev)
+ continue;
+ vma_prev = vma;
+
+ if (!is_vm_hugetlb_page(vma))
+ umem->hugetlb = 0;
+
+ if (!vma_is_dax(vma))
+ continue;
+
+ vm_ops = vma->vm_ops;
+ if (!vm_ops->lease_direct) {
+ dev_info(dma_dev, "DAX-RDMA requires a MAP_DIRECT mapping\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+
+ if (!umem->iommu)
+ umem->iommu = dma_get_iommu_domain(dma_dev);
+ if (!umem->iommu) {
+ dev_info(dma_dev, "DAX-RDMA requires an iommu protected device\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+ ld = vm_ops->lease_direct(vma, ib_umem_lease_break,
+ umem);
+ if (IS_ERR(ld)) {
+ ret = PTR_ERR(ld);
+ goto out;
+ }
+ list_add(&ld->list, &umem->leases);
}
/* preparing for next loop */
@@ -242,6 +302,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
free_page((unsigned long) page_list);
return ret < 0 ? ERR_PTR(ret) : umem;
+err_vmalist:
+ free_page((unsigned long) page_list);
+err_pagelist:
+ put_pid(umem->pid);
+ kfree(umem);
+ return ERR_PTR(-ENOMEM);
}
EXPORT_SYMBOL(ib_umem_get);
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 23159dd5be18..5048be012f96 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -34,6 +34,7 @@
#define IB_UMEM_H
#include <linux/list.h>
+#include <linux/iommu.h>
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
@@ -55,6 +56,13 @@ struct ib_umem {
struct sg_table sg_head;
int nmap;
int npages;
+ /*
+ * Note: no lock protects this list since we assume memory
+ * registration never races unregistration for a given ib_umem
+ * instance.
+ */
+ struct list_head leases;
+ struct iommu_domain *iommu;
};
/* Returns the offset of the umem start relative to the first page. */
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: Sean Hefty, linux-xfs, Jan Kara, Ashok Raj, Darrick J. Wong,
linux-rdma, linux-api, Joerg Roedel, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
Currently the ibverbs core in the kernel is completely unaware of the
dangers of filesystem-DAX mappings. Specifically, the filesystem is free
to move file blocks at will. In the case of DAX, it means that RDMA to a
given file offset can dynamically switch to another file offset, another
file, or free space with no notification to RDMA device to cease
operations. Historically, this lack of communication between the ibverbs
core and filesystem was not a problem because RDMA always targeted
dynamically allocated page cache, so at least the RDMA device would have
valid memory to target even if the file was being modified. With DAX we
need to add coordination since RDMA is bypassing page-cache and going
direct to on-media pages of the file. RDMA to DAX can cause damage if
filesystem blocks move / change state.
Use the new ->lease_direct() operation to get a notification when the
filesystem is invalidating the block map of the file and needs RDMA
operations to stop. Given that the kernel can not be in a position where
it needs to wait indefinitely for userspace to stop a device we need a
mechanism where the kernel can force-revoke access. Towards that end, use
the dma_get_iommu_domain() to both check if the device has domain
mappings that can be invalidated and retrieve the iommu_domain for use
with iommu_unmap.
Once we have that assurance that we can block in-flight I/O when the
file's block map changes then we can safely allow RDMA to DAX.
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Doug Ledford <dledford@redhat.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/infiniband/core/umem.c | 90 +++++++++++++++++++++++++++++++++++-----
include/rdma/ib_umem.h | 8 ++++
2 files changed, 86 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 21e60b1e2ff4..5e4598982359 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -36,6 +36,7 @@
#include <linux/dma-mapping.h>
#include <linux/sched/signal.h>
#include <linux/sched/mm.h>
+#include <linux/mapdirect.h>
#include <linux/export.h>
#include <linux/hugetlb.h>
#include <linux/slab.h>
@@ -46,10 +47,16 @@
static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty)
{
+ struct lease_direct *ld, *_ld;
struct scatterlist *sg;
struct page *page;
int i;
+ list_for_each_entry_safe(ld, _ld, &umem->leases, list) {
+ list_del_init(&ld->list);
+ map_direct_lease_destroy(ld);
+ }
+
if (umem->nmap > 0)
ib_dma_unmap_sg(dev, umem->sg_head.sgl,
umem->npages,
@@ -64,10 +71,20 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
}
sg_free_table(&umem->sg_head);
- return;
}
+static void ib_umem_lease_break(void *__umem)
+{
+ struct ib_umem *umem = umem;
+ struct ib_device *idev = umem->context->device;
+ struct device *dev = idev->dma_device;
+ struct scatterlist *sgl = umem->sg_head.sgl;
+
+ iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
+ iommu_sg_num_pages(dev, sgl, umem->npages));
+}
+
/**
* ib_umem_get - Pin and DMA map userspace memory.
*
@@ -96,7 +113,10 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
struct scatterlist *sg, *sg_list_start;
int need_release = 0;
unsigned int gup_flags = FOLL_WRITE;
+ struct vm_area_struct *vma_prev = NULL;
+ struct device *dma_dev;
+ dma_dev = context->device->dma_device;
if (dmasync)
dma_attrs |= DMA_ATTR_WRITE_BARRIER;
@@ -120,6 +140,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->address = addr;
umem->page_shift = PAGE_SHIFT;
umem->pid = get_task_pid(current, PIDTYPE_PID);
+ INIT_LIST_HEAD(&umem->leases);
/*
* We ask for writable memory if any of the following
* access flags are set. "Local write" and "remote write"
@@ -147,19 +168,21 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->hugetlb = 1;
page_list = (struct page **) __get_free_page(GFP_KERNEL);
- if (!page_list) {
- put_pid(umem->pid);
- kfree(umem);
- return ERR_PTR(-ENOMEM);
- }
+ if (!page_list)
+ goto err_pagelist;
/*
- * if we can't alloc the vma_list, it's not so bad;
- * just assume the memory is not hugetlb memory
+ * If DAX is enabled we need the vma to setup a ->lease_direct()
+ * lease to protect against file modifications, otherwise we can
+ * tolerate a failure to allocate the vma_list and just assume
+ * that all vmas are not hugetlb-vmas.
*/
vma_list = (struct vm_area_struct **) __get_free_page(GFP_KERNEL);
- if (!vma_list)
+ if (!vma_list) {
+ if (IS_ENABLED(CONFIG_DAX_MAP_DIRECT))
+ goto err_vmalist;
umem->hugetlb = 0;
+ }
npages = ib_umem_num_pages(umem);
@@ -199,15 +222,52 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
if (ret < 0)
goto out;
- umem->npages += ret;
cur_base += ret * PAGE_SIZE;
npages -= ret;
for_each_sg(sg_list_start, sg, ret, i) {
- if (vma_list && !is_vm_hugetlb_page(vma_list[i]))
- umem->hugetlb = 0;
+ const struct vm_operations_struct *vm_ops;
+ struct vm_area_struct *vma;
+ struct lease_direct *ld;
sg_set_page(sg, page_list[i], PAGE_SIZE, 0);
+ umem->npages++;
+
+ if (!vma_list)
+ continue;
+ vma = vma_list[i];
+
+ if (vma == vma_prev)
+ continue;
+ vma_prev = vma;
+
+ if (!is_vm_hugetlb_page(vma))
+ umem->hugetlb = 0;
+
+ if (!vma_is_dax(vma))
+ continue;
+
+ vm_ops = vma->vm_ops;
+ if (!vm_ops->lease_direct) {
+ dev_info(dma_dev, "DAX-RDMA requires a MAP_DIRECT mapping\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+
+ if (!umem->iommu)
+ umem->iommu = dma_get_iommu_domain(dma_dev);
+ if (!umem->iommu) {
+ dev_info(dma_dev, "DAX-RDMA requires an iommu protected device\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+ ld = vm_ops->lease_direct(vma, ib_umem_lease_break,
+ umem);
+ if (IS_ERR(ld)) {
+ ret = PTR_ERR(ld);
+ goto out;
+ }
+ list_add(&ld->list, &umem->leases);
}
/* preparing for next loop */
@@ -242,6 +302,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
free_page((unsigned long) page_list);
return ret < 0 ? ERR_PTR(ret) : umem;
+err_vmalist:
+ free_page((unsigned long) page_list);
+err_pagelist:
+ put_pid(umem->pid);
+ kfree(umem);
+ return ERR_PTR(-ENOMEM);
}
EXPORT_SYMBOL(ib_umem_get);
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 23159dd5be18..5048be012f96 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -34,6 +34,7 @@
#define IB_UMEM_H
#include <linux/list.h>
+#include <linux/iommu.h>
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
@@ -55,6 +56,13 @@ struct ib_umem {
struct sg_table sg_head;
int nmap;
int npages;
+ /*
+ * Note: no lock protects this list since we assume memory
+ * registration never races unregistration for a given ib_umem
+ * instance.
+ */
+ struct list_head leases;
+ struct iommu_domain *iommu;
};
/* Returns the offset of the umem start relative to the first page. */
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: Sean Hefty, linux-xfs, Jan Kara, Ashok Raj, Darrick J. Wong,
linux-rdma, linux-api, Joerg Roedel, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
Currently the ibverbs core in the kernel is completely unaware of the
dangers of filesystem-DAX mappings. Specifically, the filesystem is free
to move file blocks at will. In the case of DAX, it means that RDMA to a
given file offset can dynamically switch to another file offset, another
file, or free space with no notification to RDMA device to cease
operations. Historically, this lack of communication between the ibverbs
core and filesystem was not a problem because RDMA always targeted
dynamically allocated page cache, so at least the RDMA device would have
valid memory to target even if the file was being modified. With DAX we
need to add coordination since RDMA is bypassing page-cache and going
direct to on-media pages of the file. RDMA to DAX can cause damage if
filesystem blocks move / change state.
Use the new ->lease_direct() operation to get a notification when the
filesystem is invalidating the block map of the file and needs RDMA
operations to stop. Given that the kernel can not be in a position where
it needs to wait indefinitely for userspace to stop a device we need a
mechanism where the kernel can force-revoke access. Towards that end, use
the dma_get_iommu_domain() to both check if the device has domain
mappings that can be invalidated and retrieve the iommu_domain for use
with iommu_unmap.
Once we have that assurance that we can block in-flight I/O when the
file's block map changes then we can safely allow RDMA to DAX.
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Doug Ledford <dledford@redhat.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: Jan Kara <jack@suse.cz>
Cc: Jeff Moyer <jmoyer@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Jeff Layton <jlayton@poochiereds.net>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/infiniband/core/umem.c | 90 +++++++++++++++++++++++++++++++++++-----
include/rdma/ib_umem.h | 8 ++++
2 files changed, 86 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 21e60b1e2ff4..5e4598982359 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -36,6 +36,7 @@
#include <linux/dma-mapping.h>
#include <linux/sched/signal.h>
#include <linux/sched/mm.h>
+#include <linux/mapdirect.h>
#include <linux/export.h>
#include <linux/hugetlb.h>
#include <linux/slab.h>
@@ -46,10 +47,16 @@
static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty)
{
+ struct lease_direct *ld, *_ld;
struct scatterlist *sg;
struct page *page;
int i;
+ list_for_each_entry_safe(ld, _ld, &umem->leases, list) {
+ list_del_init(&ld->list);
+ map_direct_lease_destroy(ld);
+ }
+
if (umem->nmap > 0)
ib_dma_unmap_sg(dev, umem->sg_head.sgl,
umem->npages,
@@ -64,10 +71,20 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
}
sg_free_table(&umem->sg_head);
- return;
}
+static void ib_umem_lease_break(void *__umem)
+{
+ struct ib_umem *umem = umem;
+ struct ib_device *idev = umem->context->device;
+ struct device *dev = idev->dma_device;
+ struct scatterlist *sgl = umem->sg_head.sgl;
+
+ iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
+ iommu_sg_num_pages(dev, sgl, umem->npages));
+}
+
/**
* ib_umem_get - Pin and DMA map userspace memory.
*
@@ -96,7 +113,10 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
struct scatterlist *sg, *sg_list_start;
int need_release = 0;
unsigned int gup_flags = FOLL_WRITE;
+ struct vm_area_struct *vma_prev = NULL;
+ struct device *dma_dev;
+ dma_dev = context->device->dma_device;
if (dmasync)
dma_attrs |= DMA_ATTR_WRITE_BARRIER;
@@ -120,6 +140,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->address = addr;
umem->page_shift = PAGE_SHIFT;
umem->pid = get_task_pid(current, PIDTYPE_PID);
+ INIT_LIST_HEAD(&umem->leases);
/*
* We ask for writable memory if any of the following
* access flags are set. "Local write" and "remote write"
@@ -147,19 +168,21 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
umem->hugetlb = 1;
page_list = (struct page **) __get_free_page(GFP_KERNEL);
- if (!page_list) {
- put_pid(umem->pid);
- kfree(umem);
- return ERR_PTR(-ENOMEM);
- }
+ if (!page_list)
+ goto err_pagelist;
/*
- * if we can't alloc the vma_list, it's not so bad;
- * just assume the memory is not hugetlb memory
+ * If DAX is enabled we need the vma to setup a ->lease_direct()
+ * lease to protect against file modifications, otherwise we can
+ * tolerate a failure to allocate the vma_list and just assume
+ * that all vmas are not hugetlb-vmas.
*/
vma_list = (struct vm_area_struct **) __get_free_page(GFP_KERNEL);
- if (!vma_list)
+ if (!vma_list) {
+ if (IS_ENABLED(CONFIG_DAX_MAP_DIRECT))
+ goto err_vmalist;
umem->hugetlb = 0;
+ }
npages = ib_umem_num_pages(umem);
@@ -199,15 +222,52 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
if (ret < 0)
goto out;
- umem->npages += ret;
cur_base += ret * PAGE_SIZE;
npages -= ret;
for_each_sg(sg_list_start, sg, ret, i) {
- if (vma_list && !is_vm_hugetlb_page(vma_list[i]))
- umem->hugetlb = 0;
+ const struct vm_operations_struct *vm_ops;
+ struct vm_area_struct *vma;
+ struct lease_direct *ld;
sg_set_page(sg, page_list[i], PAGE_SIZE, 0);
+ umem->npages++;
+
+ if (!vma_list)
+ continue;
+ vma = vma_list[i];
+
+ if (vma == vma_prev)
+ continue;
+ vma_prev = vma;
+
+ if (!is_vm_hugetlb_page(vma))
+ umem->hugetlb = 0;
+
+ if (!vma_is_dax(vma))
+ continue;
+
+ vm_ops = vma->vm_ops;
+ if (!vm_ops->lease_direct) {
+ dev_info(dma_dev, "DAX-RDMA requires a MAP_DIRECT mapping\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+
+ if (!umem->iommu)
+ umem->iommu = dma_get_iommu_domain(dma_dev);
+ if (!umem->iommu) {
+ dev_info(dma_dev, "DAX-RDMA requires an iommu protected device\n");
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+ ld = vm_ops->lease_direct(vma, ib_umem_lease_break,
+ umem);
+ if (IS_ERR(ld)) {
+ ret = PTR_ERR(ld);
+ goto out;
+ }
+ list_add(&ld->list, &umem->leases);
}
/* preparing for next loop */
@@ -242,6 +302,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
free_page((unsigned long) page_list);
return ret < 0 ? ERR_PTR(ret) : umem;
+err_vmalist:
+ free_page((unsigned long) page_list);
+err_pagelist:
+ put_pid(umem->pid);
+ kfree(umem);
+ return ERR_PTR(-ENOMEM);
}
EXPORT_SYMBOL(ib_umem_get);
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 23159dd5be18..5048be012f96 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -34,6 +34,7 @@
#define IB_UMEM_H
#include <linux/list.h>
+#include <linux/iommu.h>
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
@@ -55,6 +56,13 @@ struct ib_umem {
struct sg_table sg_head;
int nmap;
int npages;
+ /*
+ * Note: no lock protects this list since we assume memory
+ * registration never races unregistration for a given ib_umem
+ * instance.
+ */
+ struct list_head leases;
+ struct iommu_domain *iommu;
};
/* Returns the offset of the umem start relative to the first page. */
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
2017-10-10 14:50 ` Dan Williams
@ 2017-10-11 11:54 ` Joerg Roedel
-1 siblings, 0 replies; 77+ messages in thread
From: Joerg Roedel @ 2017-10-11 11:54 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Sean Hefty, linux-xfs, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, linux-api, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
> +static void ib_umem_lease_break(void *__umem)
> +{
> + struct ib_umem *umem = umem;
> + struct ib_device *idev = umem->context->device;
> + struct device *dev = idev->dma_device;
> + struct scatterlist *sgl = umem->sg_head.sgl;
> +
> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
> + iommu_sg_num_pages(dev, sgl, umem->npages));
> +}
This looks like an invitation to break your code by random iommu-driver
changes. There is no guarantee that an iommu-backed dma-api
implemenation will map exactly iommu_sg_num_pages() pages for a given
sg-list. In other words, you are mixing the use of the IOMMU-API and the
DMA-API in an incompatible way that only works because you know the
internals of the iommu-drivers.
I've seen in another patch that your changes strictly require an IOMMU,
so you what you should do instead is to switch from the DMA-API to the
IOMMU-API and do the address-space management yourself.
Regards,
Joerg
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-11 11:54 ` Joerg Roedel
0 siblings, 0 replies; 77+ messages in thread
From: Joerg Roedel @ 2017-10-11 11:54 UTC (permalink / raw)
To: Dan Williams
Cc: linux-nvdimm, Sean Hefty, linux-xfs, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, linux-api, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
> +static void ib_umem_lease_break(void *__umem)
> +{
> + struct ib_umem *umem = umem;
> + struct ib_device *idev = umem->context->device;
> + struct device *dev = idev->dma_device;
> + struct scatterlist *sgl = umem->sg_head.sgl;
> +
> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
> + iommu_sg_num_pages(dev, sgl, umem->npages));
> +}
This looks like an invitation to break your code by random iommu-driver
changes. There is no guarantee that an iommu-backed dma-api
implemenation will map exactly iommu_sg_num_pages() pages for a given
sg-list. In other words, you are mixing the use of the IOMMU-API and the
DMA-API in an incompatible way that only works because you know the
internals of the iommu-drivers.
I've seen in another patch that your changes strictly require an IOMMU,
so you what you should do instead is to switch from the DMA-API to the
IOMMU-API and do the address-space management yourself.
Regards,
Joerg
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-11 16:01 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 16:01 UTC (permalink / raw)
To: Joerg Roedel
Cc: J. Bruce Fields, Doug Ledford, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, Linux API, linux-nvdimm,
Dave Chinner, iommu, Hal Rosenstock, linux-xfs, linux-mm,
Jeff Layton, linux-fsdevel, Sean Hefty, David Woodhouse,
Christoph Hellwig
On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel <joro@8bytes.org> wrote:
> On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
>> +static void ib_umem_lease_break(void *__umem)
>> +{
>> + struct ib_umem *umem = umem;
>> + struct ib_device *idev = umem->context->device;
>> + struct device *dev = idev->dma_device;
>> + struct scatterlist *sgl = umem->sg_head.sgl;
>> +
>> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
>> + iommu_sg_num_pages(dev, sgl, umem->npages));
>> +}
>
> This looks like an invitation to break your code by random iommu-driver
> changes. There is no guarantee that an iommu-backed dma-api
> implemenation will map exactly iommu_sg_num_pages() pages for a given
> sg-list. In other words, you are mixing the use of the IOMMU-API and the
> DMA-API in an incompatible way that only works because you know the
> internals of the iommu-drivers.
>
> I've seen in another patch that your changes strictly require an IOMMU,
> so you what you should do instead is to switch from the DMA-API to the
> IOMMU-API and do the address-space management yourself.
>
Ok, I'll switch over completely to the iommu api for this. It will
also address Robin's concern.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-11 16:01 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 16:01 UTC (permalink / raw)
To: Joerg Roedel
Cc: linux-nvdimm, Sean Hefty, linux-xfs, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, Linux API, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel <joro@8bytes.org> wrote:
> On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
>> +static void ib_umem_lease_break(void *__umem)
>> +{
>> + struct ib_umem *umem = umem;
>> + struct ib_device *idev = umem->context->device;
>> + struct device *dev = idev->dma_device;
>> + struct scatterlist *sgl = umem->sg_head.sgl;
>> +
>> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
>> + iommu_sg_num_pages(dev, sgl, umem->npages));
>> +}
>
> This looks like an invitation to break your code by random iommu-driver
> changes. There is no guarantee that an iommu-backed dma-api
> implemenation will map exactly iommu_sg_num_pages() pages for a given
> sg-list. In other words, you are mixing the use of the IOMMU-API and the
> DMA-API in an incompatible way that only works because you know the
> internals of the iommu-drivers.
>
> I've seen in another patch that your changes strictly require an IOMMU,
> so you what you should do instead is to switch from the DMA-API to the
> IOMMU-API and do the address-space management yourself.
>
Ok, I'll switch over completely to the iommu api for this. It will
also address Robin's concern.
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-11 16:01 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 16:01 UTC (permalink / raw)
To: Joerg Roedel
Cc: linux-nvdimm, Sean Hefty, linux-xfs, Jan Kara, Ashok Raj,
Darrick J. Wong, linux-rdma, Linux API, Dave Chinner, Jeff Moyer,
iommu, Christoph Hellwig, J. Bruce Fields, linux-mm,
Doug Ledford, Ross Zwisler, linux-fsdevel, Jeff Layton,
David Woodhouse, Hal Rosenstock
On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel <joro@8bytes.org> wrote:
> On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
>> +static void ib_umem_lease_break(void *__umem)
>> +{
>> + struct ib_umem *umem = umem;
>> + struct ib_device *idev = umem->context->device;
>> + struct device *dev = idev->dma_device;
>> + struct scatterlist *sgl = umem->sg_head.sgl;
>> +
>> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
>> + iommu_sg_num_pages(dev, sgl, umem->npages));
>> +}
>
> This looks like an invitation to break your code by random iommu-driver
> changes. There is no guarantee that an iommu-backed dma-api
> implemenation will map exactly iommu_sg_num_pages() pages for a given
> sg-list. In other words, you are mixing the use of the IOMMU-API and the
> DMA-API in an incompatible way that only works because you know the
> internals of the iommu-drivers.
>
> I've seen in another patch that your changes strictly require an IOMMU,
> so you what you should do instead is to switch from the DMA-API to the
> IOMMU-API and do the address-space management yourself.
>
Ok, I'll switch over completely to the iommu api for this. It will
also address Robin's concern.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings
@ 2017-10-11 16:01 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-11 16:01 UTC (permalink / raw)
To: Joerg Roedel
Cc: J. Bruce Fields, Doug Ledford, Jan Kara, Darrick J. Wong,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Linux API,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw, Dave Chinner,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
Hal Rosenstock, linux-xfs-u79uwXL29TY76Z2rM5mHXA, linux-mm,
Jeff Moyer, Jeff Layton, Ross Zwisler, linux-fsdevel, Sean Hefty,
David Woodhouse, Christoph Hellwig
On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> wrote:
> On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote:
>> +static void ib_umem_lease_break(void *__umem)
>> +{
>> + struct ib_umem *umem = umem;
>> + struct ib_device *idev = umem->context->device;
>> + struct device *dev = idev->dma_device;
>> + struct scatterlist *sgl = umem->sg_head.sgl;
>> +
>> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK,
>> + iommu_sg_num_pages(dev, sgl, umem->npages));
>> +}
>
> This looks like an invitation to break your code by random iommu-driver
> changes. There is no guarantee that an iommu-backed dma-api
> implemenation will map exactly iommu_sg_num_pages() pages for a given
> sg-list. In other words, you are mixing the use of the IOMMU-API and the
> DMA-API in an incompatible way that only works because you know the
> internals of the iommu-drivers.
>
> I've seen in another patch that your changes strictly require an IOMMU,
> so you what you should do instead is to switch from the DMA-API to the
> IOMMU-API and do the address-space management yourself.
>
Ok, I'll switch over completely to the iommu api for this. It will
also address Robin's concern.
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 14/14] tools/testing/nvdimm: enable rdma unit tests
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, linux-xfs, linux-mm, iommu, linux-fsdevel
Provide a mock dma_get_iommu_domain() for the ibverbs core. Enable
ib_umem_get() to satisfy its DAX safety checks for a controlled test.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
tools/testing/nvdimm/Kbuild | 31 +++++++++++++++++++++++++++++++
tools/testing/nvdimm/config_check.c | 2 ++
tools/testing/nvdimm/test/iomap.c | 14 ++++++++++++++
3 files changed, 47 insertions(+)
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
index d870520da68b..f4a007090950 100644
--- a/tools/testing/nvdimm/Kbuild
+++ b/tools/testing/nvdimm/Kbuild
@@ -15,11 +15,13 @@ ldflags-y += --wrap=insert_resource
ldflags-y += --wrap=remove_resource
ldflags-y += --wrap=acpi_evaluate_object
ldflags-y += --wrap=acpi_evaluate_dsm
+ldflags-y += --wrap=dma_get_iommu_domain
DRIVERS := ../../../drivers
NVDIMM_SRC := $(DRIVERS)/nvdimm
ACPI_SRC := $(DRIVERS)/acpi/nfit
DAX_SRC := $(DRIVERS)/dax
+IBCORE := $(DRIVERS)/infiniband/core
ccflags-y := -I$(src)/$(NVDIMM_SRC)/
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
@@ -33,6 +35,7 @@ obj-$(CONFIG_DAX) += dax.o
endif
obj-$(CONFIG_DEV_DAX) += device_dax.o
obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
+obj-$(CONFIG_INFINIBAND) += ib_core.o
nfit-y := $(ACPI_SRC)/core.o
nfit-$(CONFIG_X86_MCE) += $(ACPI_SRC)/mce.o
@@ -75,4 +78,32 @@ libnvdimm-$(CONFIG_NVDIMM_PFN) += $(NVDIMM_SRC)/pfn_devs.o
libnvdimm-$(CONFIG_NVDIMM_DAX) += $(NVDIMM_SRC)/dax_devs.o
libnvdimm-y += config_check.o
+ib_core-y := $(IBCORE)/packer.o
+ib_core-y += $(IBCORE)/ud_header.o
+ib_core-y += $(IBCORE)/verbs.o
+ib_core-y += $(IBCORE)/cq.o
+ib_core-y += $(IBCORE)/rw.o
+ib_core-y += $(IBCORE)/sysfs.o
+ib_core-y += $(IBCORE)/device.o
+ib_core-y += $(IBCORE)/fmr_pool.o
+ib_core-y += $(IBCORE)/cache.o
+ib_core-y += $(IBCORE)/netlink.o
+ib_core-y += $(IBCORE)/roce_gid_mgmt.o
+ib_core-y += $(IBCORE)/mr_pool.o
+ib_core-y += $(IBCORE)/addr.o
+ib_core-y += $(IBCORE)/sa_query.o
+ib_core-y += $(IBCORE)/multicast.o
+ib_core-y += $(IBCORE)/mad.o
+ib_core-y += $(IBCORE)/smi.o
+ib_core-y += $(IBCORE)/agent.o
+ib_core-y += $(IBCORE)/mad_rmpp.o
+ib_core-y += $(IBCORE)/security.o
+ib_core-y += $(IBCORE)/nldev.o
+
+ib_core-$(CONFIG_INFINIBAND_USER_MEM) += $(IBCORE)/umem.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_odp.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_rbtree.o
+ib_core-$(CONFIG_CGROUP_RDMA) += $(IBCORE)/cgroup.o
+ib_core-y += config_check.o
+
obj-m += test/
diff --git a/tools/testing/nvdimm/config_check.c b/tools/testing/nvdimm/config_check.c
index 7dc5a0af9b54..33e7c805bfd6 100644
--- a/tools/testing/nvdimm/config_check.c
+++ b/tools/testing/nvdimm/config_check.c
@@ -14,4 +14,6 @@ void check(void)
BUILD_BUG_ON(!IS_MODULE(CONFIG_ACPI_NFIT));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX_PMEM));
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_INFINIBAND));
}
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
index e1f75a1914a1..1e439b2b01e7 100644
--- a/tools/testing/nvdimm/test/iomap.c
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -17,6 +17,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/pfn_t.h>
+#include <linux/iommu.h>
#include <linux/acpi.h>
#include <linux/io.h>
#include <linux/mm.h>
@@ -388,4 +389,17 @@ union acpi_object * __wrap_acpi_evaluate_dsm(acpi_handle handle, const guid_t *g
}
EXPORT_SYMBOL(__wrap_acpi_evaluate_dsm);
+/*
+ * This assumes that any iommu api routine we would call with this
+ * domain checks for NULL ops and either returns an error or does
+ * nothing.
+ */
+struct iommu_domain *__wrap_dma_get_iommu_domain(struct device *dev)
+{
+ static struct iommu_domain domain;
+
+ return &domain;
+}
+EXPORT_SYMBOL(__wrap_dma_get_iommu_domain);
+
MODULE_LICENSE("GPL v2");
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 14/14] tools/testing/nvdimm: enable rdma unit tests
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, linux-xfs, linux-mm, iommu, linux-fsdevel
Provide a mock dma_get_iommu_domain() for the ibverbs core. Enable
ib_umem_get() to satisfy its DAX safety checks for a controlled test.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
tools/testing/nvdimm/Kbuild | 31 +++++++++++++++++++++++++++++++
tools/testing/nvdimm/config_check.c | 2 ++
tools/testing/nvdimm/test/iomap.c | 14 ++++++++++++++
3 files changed, 47 insertions(+)
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
index d870520da68b..f4a007090950 100644
--- a/tools/testing/nvdimm/Kbuild
+++ b/tools/testing/nvdimm/Kbuild
@@ -15,11 +15,13 @@ ldflags-y += --wrap=insert_resource
ldflags-y += --wrap=remove_resource
ldflags-y += --wrap=acpi_evaluate_object
ldflags-y += --wrap=acpi_evaluate_dsm
+ldflags-y += --wrap=dma_get_iommu_domain
DRIVERS := ../../../drivers
NVDIMM_SRC := $(DRIVERS)/nvdimm
ACPI_SRC := $(DRIVERS)/acpi/nfit
DAX_SRC := $(DRIVERS)/dax
+IBCORE := $(DRIVERS)/infiniband/core
ccflags-y := -I$(src)/$(NVDIMM_SRC)/
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
@@ -33,6 +35,7 @@ obj-$(CONFIG_DAX) += dax.o
endif
obj-$(CONFIG_DEV_DAX) += device_dax.o
obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
+obj-$(CONFIG_INFINIBAND) += ib_core.o
nfit-y := $(ACPI_SRC)/core.o
nfit-$(CONFIG_X86_MCE) += $(ACPI_SRC)/mce.o
@@ -75,4 +78,32 @@ libnvdimm-$(CONFIG_NVDIMM_PFN) += $(NVDIMM_SRC)/pfn_devs.o
libnvdimm-$(CONFIG_NVDIMM_DAX) += $(NVDIMM_SRC)/dax_devs.o
libnvdimm-y += config_check.o
+ib_core-y := $(IBCORE)/packer.o
+ib_core-y += $(IBCORE)/ud_header.o
+ib_core-y += $(IBCORE)/verbs.o
+ib_core-y += $(IBCORE)/cq.o
+ib_core-y += $(IBCORE)/rw.o
+ib_core-y += $(IBCORE)/sysfs.o
+ib_core-y += $(IBCORE)/device.o
+ib_core-y += $(IBCORE)/fmr_pool.o
+ib_core-y += $(IBCORE)/cache.o
+ib_core-y += $(IBCORE)/netlink.o
+ib_core-y += $(IBCORE)/roce_gid_mgmt.o
+ib_core-y += $(IBCORE)/mr_pool.o
+ib_core-y += $(IBCORE)/addr.o
+ib_core-y += $(IBCORE)/sa_query.o
+ib_core-y += $(IBCORE)/multicast.o
+ib_core-y += $(IBCORE)/mad.o
+ib_core-y += $(IBCORE)/smi.o
+ib_core-y += $(IBCORE)/agent.o
+ib_core-y += $(IBCORE)/mad_rmpp.o
+ib_core-y += $(IBCORE)/security.o
+ib_core-y += $(IBCORE)/nldev.o
+
+ib_core-$(CONFIG_INFINIBAND_USER_MEM) += $(IBCORE)/umem.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_odp.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_rbtree.o
+ib_core-$(CONFIG_CGROUP_RDMA) += $(IBCORE)/cgroup.o
+ib_core-y += config_check.o
+
obj-m += test/
diff --git a/tools/testing/nvdimm/config_check.c b/tools/testing/nvdimm/config_check.c
index 7dc5a0af9b54..33e7c805bfd6 100644
--- a/tools/testing/nvdimm/config_check.c
+++ b/tools/testing/nvdimm/config_check.c
@@ -14,4 +14,6 @@ void check(void)
BUILD_BUG_ON(!IS_MODULE(CONFIG_ACPI_NFIT));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX_PMEM));
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_INFINIBAND));
}
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
index e1f75a1914a1..1e439b2b01e7 100644
--- a/tools/testing/nvdimm/test/iomap.c
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -17,6 +17,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/pfn_t.h>
+#include <linux/iommu.h>
#include <linux/acpi.h>
#include <linux/io.h>
#include <linux/mm.h>
@@ -388,4 +389,17 @@ union acpi_object * __wrap_acpi_evaluate_dsm(acpi_handle handle, const guid_t *g
}
EXPORT_SYMBOL(__wrap_acpi_evaluate_dsm);
+/*
+ * This assumes that any iommu api routine we would call with this
+ * domain checks for NULL ops and either returns an error or does
+ * nothing.
+ */
+struct iommu_domain *__wrap_dma_get_iommu_domain(struct device *dev)
+{
+ static struct iommu_domain domain;
+
+ return &domain;
+}
+EXPORT_SYMBOL(__wrap_dma_get_iommu_domain);
+
MODULE_LICENSE("GPL v2");
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 14/14] tools/testing/nvdimm: enable rdma unit tests
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm
Cc: linux-rdma, linux-api, linux-xfs, linux-mm, iommu, linux-fsdevel
Provide a mock dma_get_iommu_domain() for the ibverbs core. Enable
ib_umem_get() to satisfy its DAX safety checks for a controlled test.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
tools/testing/nvdimm/Kbuild | 31 +++++++++++++++++++++++++++++++
tools/testing/nvdimm/config_check.c | 2 ++
tools/testing/nvdimm/test/iomap.c | 14 ++++++++++++++
3 files changed, 47 insertions(+)
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
index d870520da68b..f4a007090950 100644
--- a/tools/testing/nvdimm/Kbuild
+++ b/tools/testing/nvdimm/Kbuild
@@ -15,11 +15,13 @@ ldflags-y += --wrap=insert_resource
ldflags-y += --wrap=remove_resource
ldflags-y += --wrap=acpi_evaluate_object
ldflags-y += --wrap=acpi_evaluate_dsm
+ldflags-y += --wrap=dma_get_iommu_domain
DRIVERS := ../../../drivers
NVDIMM_SRC := $(DRIVERS)/nvdimm
ACPI_SRC := $(DRIVERS)/acpi/nfit
DAX_SRC := $(DRIVERS)/dax
+IBCORE := $(DRIVERS)/infiniband/core
ccflags-y := -I$(src)/$(NVDIMM_SRC)/
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
@@ -33,6 +35,7 @@ obj-$(CONFIG_DAX) += dax.o
endif
obj-$(CONFIG_DEV_DAX) += device_dax.o
obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
+obj-$(CONFIG_INFINIBAND) += ib_core.o
nfit-y := $(ACPI_SRC)/core.o
nfit-$(CONFIG_X86_MCE) += $(ACPI_SRC)/mce.o
@@ -75,4 +78,32 @@ libnvdimm-$(CONFIG_NVDIMM_PFN) += $(NVDIMM_SRC)/pfn_devs.o
libnvdimm-$(CONFIG_NVDIMM_DAX) += $(NVDIMM_SRC)/dax_devs.o
libnvdimm-y += config_check.o
+ib_core-y := $(IBCORE)/packer.o
+ib_core-y += $(IBCORE)/ud_header.o
+ib_core-y += $(IBCORE)/verbs.o
+ib_core-y += $(IBCORE)/cq.o
+ib_core-y += $(IBCORE)/rw.o
+ib_core-y += $(IBCORE)/sysfs.o
+ib_core-y += $(IBCORE)/device.o
+ib_core-y += $(IBCORE)/fmr_pool.o
+ib_core-y += $(IBCORE)/cache.o
+ib_core-y += $(IBCORE)/netlink.o
+ib_core-y += $(IBCORE)/roce_gid_mgmt.o
+ib_core-y += $(IBCORE)/mr_pool.o
+ib_core-y += $(IBCORE)/addr.o
+ib_core-y += $(IBCORE)/sa_query.o
+ib_core-y += $(IBCORE)/multicast.o
+ib_core-y += $(IBCORE)/mad.o
+ib_core-y += $(IBCORE)/smi.o
+ib_core-y += $(IBCORE)/agent.o
+ib_core-y += $(IBCORE)/mad_rmpp.o
+ib_core-y += $(IBCORE)/security.o
+ib_core-y += $(IBCORE)/nldev.o
+
+ib_core-$(CONFIG_INFINIBAND_USER_MEM) += $(IBCORE)/umem.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_odp.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_rbtree.o
+ib_core-$(CONFIG_CGROUP_RDMA) += $(IBCORE)/cgroup.o
+ib_core-y += config_check.o
+
obj-m += test/
diff --git a/tools/testing/nvdimm/config_check.c b/tools/testing/nvdimm/config_check.c
index 7dc5a0af9b54..33e7c805bfd6 100644
--- a/tools/testing/nvdimm/config_check.c
+++ b/tools/testing/nvdimm/config_check.c
@@ -14,4 +14,6 @@ void check(void)
BUILD_BUG_ON(!IS_MODULE(CONFIG_ACPI_NFIT));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX_PMEM));
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_INFINIBAND));
}
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
index e1f75a1914a1..1e439b2b01e7 100644
--- a/tools/testing/nvdimm/test/iomap.c
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -17,6 +17,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/pfn_t.h>
+#include <linux/iommu.h>
#include <linux/acpi.h>
#include <linux/io.h>
#include <linux/mm.h>
@@ -388,4 +389,17 @@ union acpi_object * __wrap_acpi_evaluate_dsm(acpi_handle handle, const guid_t *g
}
EXPORT_SYMBOL(__wrap_acpi_evaluate_dsm);
+/*
+ * This assumes that any iommu api routine we would call with this
+ * domain checks for NULL ops and either returns an error or does
+ * nothing.
+ */
+struct iommu_domain *__wrap_dma_get_iommu_domain(struct device *dev)
+{
+ static struct iommu_domain domain;
+
+ return &domain;
+}
+EXPORT_SYMBOL(__wrap_dma_get_iommu_domain);
+
MODULE_LICENSE("GPL v2");
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 14/14] tools/testing/nvdimm: enable rdma unit tests
@ 2017-10-10 14:50 ` Dan Williams
0 siblings, 0 replies; 77+ messages in thread
From: Dan Williams @ 2017-10-10 14:50 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-xfs-u79uwXL29TY76Z2rM5mHXA,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA
Provide a mock dma_get_iommu_domain() for the ibverbs core. Enable
ib_umem_get() to satisfy its DAX safety checks for a controlled test.
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
tools/testing/nvdimm/Kbuild | 31 +++++++++++++++++++++++++++++++
tools/testing/nvdimm/config_check.c | 2 ++
tools/testing/nvdimm/test/iomap.c | 14 ++++++++++++++
3 files changed, 47 insertions(+)
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
index d870520da68b..f4a007090950 100644
--- a/tools/testing/nvdimm/Kbuild
+++ b/tools/testing/nvdimm/Kbuild
@@ -15,11 +15,13 @@ ldflags-y += --wrap=insert_resource
ldflags-y += --wrap=remove_resource
ldflags-y += --wrap=acpi_evaluate_object
ldflags-y += --wrap=acpi_evaluate_dsm
+ldflags-y += --wrap=dma_get_iommu_domain
DRIVERS := ../../../drivers
NVDIMM_SRC := $(DRIVERS)/nvdimm
ACPI_SRC := $(DRIVERS)/acpi/nfit
DAX_SRC := $(DRIVERS)/dax
+IBCORE := $(DRIVERS)/infiniband/core
ccflags-y := -I$(src)/$(NVDIMM_SRC)/
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
@@ -33,6 +35,7 @@ obj-$(CONFIG_DAX) += dax.o
endif
obj-$(CONFIG_DEV_DAX) += device_dax.o
obj-$(CONFIG_DEV_DAX_PMEM) += dax_pmem.o
+obj-$(CONFIG_INFINIBAND) += ib_core.o
nfit-y := $(ACPI_SRC)/core.o
nfit-$(CONFIG_X86_MCE) += $(ACPI_SRC)/mce.o
@@ -75,4 +78,32 @@ libnvdimm-$(CONFIG_NVDIMM_PFN) += $(NVDIMM_SRC)/pfn_devs.o
libnvdimm-$(CONFIG_NVDIMM_DAX) += $(NVDIMM_SRC)/dax_devs.o
libnvdimm-y += config_check.o
+ib_core-y := $(IBCORE)/packer.o
+ib_core-y += $(IBCORE)/ud_header.o
+ib_core-y += $(IBCORE)/verbs.o
+ib_core-y += $(IBCORE)/cq.o
+ib_core-y += $(IBCORE)/rw.o
+ib_core-y += $(IBCORE)/sysfs.o
+ib_core-y += $(IBCORE)/device.o
+ib_core-y += $(IBCORE)/fmr_pool.o
+ib_core-y += $(IBCORE)/cache.o
+ib_core-y += $(IBCORE)/netlink.o
+ib_core-y += $(IBCORE)/roce_gid_mgmt.o
+ib_core-y += $(IBCORE)/mr_pool.o
+ib_core-y += $(IBCORE)/addr.o
+ib_core-y += $(IBCORE)/sa_query.o
+ib_core-y += $(IBCORE)/multicast.o
+ib_core-y += $(IBCORE)/mad.o
+ib_core-y += $(IBCORE)/smi.o
+ib_core-y += $(IBCORE)/agent.o
+ib_core-y += $(IBCORE)/mad_rmpp.o
+ib_core-y += $(IBCORE)/security.o
+ib_core-y += $(IBCORE)/nldev.o
+
+ib_core-$(CONFIG_INFINIBAND_USER_MEM) += $(IBCORE)/umem.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_odp.o
+ib_core-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += $(IBCORE)/umem_rbtree.o
+ib_core-$(CONFIG_CGROUP_RDMA) += $(IBCORE)/cgroup.o
+ib_core-y += config_check.o
+
obj-m += test/
diff --git a/tools/testing/nvdimm/config_check.c b/tools/testing/nvdimm/config_check.c
index 7dc5a0af9b54..33e7c805bfd6 100644
--- a/tools/testing/nvdimm/config_check.c
+++ b/tools/testing/nvdimm/config_check.c
@@ -14,4 +14,6 @@ void check(void)
BUILD_BUG_ON(!IS_MODULE(CONFIG_ACPI_NFIT));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX));
BUILD_BUG_ON(!IS_MODULE(CONFIG_DEV_DAX_PMEM));
+ BUILD_BUG_ON(!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM));
+ BUILD_BUG_ON(!IS_MODULE(CONFIG_INFINIBAND));
}
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
index e1f75a1914a1..1e439b2b01e7 100644
--- a/tools/testing/nvdimm/test/iomap.c
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -17,6 +17,7 @@
#include <linux/module.h>
#include <linux/types.h>
#include <linux/pfn_t.h>
+#include <linux/iommu.h>
#include <linux/acpi.h>
#include <linux/io.h>
#include <linux/mm.h>
@@ -388,4 +389,17 @@ union acpi_object * __wrap_acpi_evaluate_dsm(acpi_handle handle, const guid_t *g
}
EXPORT_SYMBOL(__wrap_acpi_evaluate_dsm);
+/*
+ * This assumes that any iommu api routine we would call with this
+ * domain checks for NULL ops and either returns an error or does
+ * nothing.
+ */
+struct iommu_domain *__wrap_dma_get_iommu_domain(struct device *dev)
+{
+ static struct iommu_domain domain;
+
+ return &domain;
+}
+EXPORT_SYMBOL(__wrap_dma_get_iommu_domain);
+
MODULE_LICENSE("GPL v2");
^ permalink raw reply related [flat|nested] 77+ messages in thread