From: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
To: akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org
Cc: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>,
Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org>,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org,
David Airlie <airlied-cv59FeDIM0c@public.gmane.org>,
linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
"Darrick J. Wong"
<darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>,
Dave Chinner <david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org>,
Takashi Iwai <tiwai-IBi9RG/b67k@public.gmane.org>,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-xfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Julia Lawall <julia.lawall-L2FTfq7BK8M@public.gmane.org>,
Alexander Viro
<viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn@public.gmane.org>,
luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Daniel Vetter
<daniel.vetter-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org
Subject: [PATCH v6 0/5] MAP_DIRECT and block-map-atomic files
Date: Wed, 23 Aug 2017 16:48:34 -0700 [thread overview]
Message-ID: <150353211413.5039.5228914877418362329.stgit@dwillia2-desk3.amr.corp.intel.com> (raw)
Changes since v5 [1]:
* Compile fixes from a much improved coccinelle semantic patch (thanks
Julia!) that adds a 'flags' argument to all the ->mmap()
implementations in the kernel. (0day-kbuild-robot)
* Make the deprecated MAP_DENYWRITE and MAP_EXECUTABLE flags return
EOPNOTSUPP with the new mmap3() syscall. (Kirill)
* Minor changelog updates.
* Updated cover letter with a clarified summary and checklist of
questions to answer before proceeding further.
---
MAP_DIRECT is a mechanism to ask the kernel to atomically manage the
file-offset to physical-address block relationship of a mapping relative
to any memory-mapped access. It is complimentary to the proposed
MAP_SYNC mechanism which makes the same guarantee relative to cpu
faults. MAP_DIRECT goes a step further and makes this guarantee for
agents that may not generate mmu faults, but at the cost of restricting
the kernel's ability to mutate the block-map at will.
MAP_SYNC is preferred for scenarios that want full filesystem feature
support while avoiding fsync/msync overhead, but also do not need to
contend with hypervisors or RDMA agents that do not give the kernel an
mmu fault. In other words, the need for MAP_DIRECT is driven by the
scarcity of SVM capable hardware (Shared Virtual Memory, where hardware
generates mmu faults), hypervisors like Xen that need to interrogate the
physical address layout of a file to maintain their own physical-address
mapping metadata outside the kernel, and peer-to-peer DMA use cases that
always bypass the mmu.
The MAP_DIRECT mechanism allows a filesystem to be used for capacity
provisioning and access control where these aforementioned applications
would otherwise be forced to roll a custom solution on top of a raw
device-file.
Questions:
1/ Is the definition of MAP_DIRECT constrained enough to allow us to
make the restrictions it imposes on the kernel finer grained over time?
2/ Do the XFS changes look sane? They attempt to avoid adding any
overhead to the non-MAP_DIRECT case at the expense of the new
i_mapdcount atomic counter in the XFS inode.
3/ While the generic MAP_DIRECT description warns that the block-map may
not be actually be immutable for the lifetime of the mapping it also
does not preclude a filesystem from making that guarantee. In fact,
Dave wants to be able to get a stable view of the physical mapping
[2], and Xen has a need to do the same [3]. Do we want userspace to
start making "XFS + MAP_DIRECT == Immutable" assumptions, or do we
need a separate mechanism for that guarantee?
[1]: https://lkml.org/lkml/2017/8/16/114
[2]: https://www.mail-archive.com/linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg1467677.html
[3]: https://lists.xen.org/archives/html/xen-devel/2017-04/msg00419.html
---
Dan Williams (5):
vfs: add flags parameter to ->mmap() in 'struct file_operations'
fs, xfs: introduce S_IOMAP_SEALED
mm: introduce mmap3 for safely defining new mmap flags
fs, xfs: introduce MAP_DIRECT for creating block-map-atomic file ranges
fs, fcntl: add F_MAP_DIRECT
arch/arc/kernel/arc_hostlink.c | 3 -
arch/mips/kernel/vdso.c | 2
arch/powerpc/kernel/proc_powerpc.c | 3 -
arch/powerpc/kvm/book3s_64_vio.c | 3 -
arch/powerpc/platforms/cell/spufs/file.c | 21 ++--
arch/powerpc/platforms/powernv/opal-prd.c | 3 -
arch/um/drivers/mmapper_kern.c | 3 -
arch/x86/entry/syscalls/syscall_32.tbl | 1
arch/x86/entry/syscalls/syscall_64.tbl | 1
drivers/android/binder.c | 3 -
drivers/char/agp/frontend.c | 3 -
drivers/char/bsr.c | 3 -
drivers/char/hpet.c | 6 +
drivers/char/mbcs.c | 3 -
drivers/char/mbcs.h | 3 -
drivers/char/mem.c | 11 +-
drivers/char/mspec.c | 9 +-
drivers/char/uv_mmtimer.c | 6 +
drivers/dax/device.c | 3 -
drivers/dma-buf/dma-buf.c | 4 +
drivers/firewire/core-cdev.c | 3 -
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 3 -
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 3 -
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 5 +
drivers/gpu/drm/arc/arcpgu_drv.c | 5 +
drivers/gpu/drm/ast/ast_drv.h | 3 -
drivers/gpu/drm/ast/ast_ttm.c | 3 -
drivers/gpu/drm/bochs/bochs.h | 3 -
drivers/gpu/drm/bochs/bochs_mm.c | 3 -
drivers/gpu/drm/cirrus/cirrus_drv.h | 3 -
drivers/gpu/drm/cirrus/cirrus_ttm.c | 3 -
drivers/gpu/drm/drm_gem.c | 3 -
drivers/gpu/drm/drm_gem_cma_helper.c | 6 +
drivers/gpu/drm/drm_vm.c | 3 -
drivers/gpu/drm/etnaviv/etnaviv_drv.h | 3 -
drivers/gpu/drm/etnaviv/etnaviv_gem.c | 5 +
drivers/gpu/drm/exynos/exynos_drm_gem.c | 5 +
drivers/gpu/drm/exynos/exynos_drm_gem.h | 3 -
drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h | 3 -
drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c | 3 -
drivers/gpu/drm/i810/i810_dma.c | 3 -
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 2
drivers/gpu/drm/mediatek/mtk_drm_gem.c | 5 +
drivers/gpu/drm/mediatek/mtk_drm_gem.h | 3 -
drivers/gpu/drm/mgag200/mgag200_drv.h | 3 -
drivers/gpu/drm/mgag200/mgag200_ttm.c | 3 -
drivers/gpu/drm/msm/msm_drv.h | 3 -
drivers/gpu/drm/msm/msm_gem.c | 5 +
drivers/gpu/drm/nouveau/nouveau_ttm.c | 5 +
drivers/gpu/drm/nouveau/nouveau_ttm.h | 2
drivers/gpu/drm/omapdrm/omap_drv.h | 3 -
drivers/gpu/drm/omapdrm/omap_gem.c | 5 +
drivers/gpu/drm/qxl/qxl_drv.h | 3 -
drivers/gpu/drm/qxl/qxl_ttm.c | 3 -
drivers/gpu/drm/radeon/radeon_drv.c | 3 -
drivers/gpu/drm/radeon/radeon_ttm.c | 3 -
drivers/gpu/drm/rockchip/rockchip_drm_gem.c | 5 +
drivers/gpu/drm/rockchip/rockchip_drm_gem.h | 3 -
drivers/gpu/drm/tegra/gem.c | 5 +
drivers/gpu/drm/tegra/gem.h | 3 -
drivers/gpu/drm/udl/udl_drv.h | 3 -
drivers/gpu/drm/udl/udl_gem.c | 5 +
drivers/gpu/drm/vc4/vc4_bo.c | 5 +
drivers/gpu/drm/vc4/vc4_drv.h | 3 -
drivers/gpu/drm/vgem/vgem_drv.c | 7 +
drivers/gpu/drm/virtio/virtgpu_drv.h | 3 -
drivers/gpu/drm/virtio/virtgpu_ttm.c | 3 -
drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 3 -
drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c | 3 -
drivers/hsi/clients/cmt_speech.c | 3 -
drivers/hwtracing/intel_th/msu.c | 3 -
drivers/hwtracing/stm/core.c | 3 -
drivers/infiniband/core/uverbs_main.c | 3 -
drivers/infiniband/hw/hfi1/file_ops.c | 6 +
drivers/infiniband/hw/qib/qib_file_ops.c | 5 +
drivers/media/v4l2-core/v4l2-dev.c | 3 -
drivers/misc/aspeed-lpc-ctrl.c | 3 -
drivers/misc/cxl/api.c | 5 +
drivers/misc/cxl/cxl.h | 3 -
drivers/misc/cxl/file.c | 3 -
drivers/misc/genwqe/card_dev.c | 3 -
drivers/misc/mic/scif/scif_fd.c | 3 -
drivers/misc/mic/vop/vop_vringh.c | 3 -
drivers/misc/sgi-gru/grufile.c | 3 -
drivers/mtd/mtdchar.c | 3 -
drivers/pci/proc.c | 3 -
drivers/rapidio/devices/rio_mport_cdev.c | 3 -
drivers/sbus/char/flash.c | 3 -
drivers/sbus/char/jsflash.c | 3 -
drivers/scsi/cxlflash/superpipe.c | 5 +
drivers/scsi/sg.c | 3 -
drivers/staging/android/ashmem.c | 3 -
drivers/staging/comedi/comedi_fops.c | 3 -
.../staging/lustre/lustre/llite/llite_internal.h | 3 -
drivers/staging/lustre/lustre/llite/llite_mmap.c | 5 +
drivers/staging/vboxvideo/vbox_drv.h | 3 -
drivers/staging/vboxvideo/vbox_ttm.c | 3 -
drivers/staging/vme/devices/vme_user.c | 3 -
drivers/uio/uio.c | 3 -
drivers/usb/core/devio.c | 3 -
drivers/usb/mon/mon_bin.c | 3 -
drivers/vfio/vfio.c | 7 +
drivers/video/fbdev/core/fbmem.c | 3 -
drivers/video/fbdev/pxa3xx-gcu.c | 3 -
drivers/xen/gntalloc.c | 3 -
drivers/xen/gntdev.c | 3 -
drivers/xen/privcmd.c | 3 -
drivers/xen/xenbus/xenbus_dev_backend.c | 3 -
drivers/xen/xenfs/xenstored.c | 3 -
fs/9p/vfs_file.c | 10 +-
fs/aio.c | 3 -
fs/attr.c | 10 ++
fs/btrfs/file.c | 3 -
fs/ceph/addr.c | 3 -
fs/ceph/super.h | 3 -
fs/cifs/cifsfs.h | 6 +
fs/cifs/file.c | 10 +-
fs/coda/file.c | 5 +
fs/ecryptfs/file.c | 5 +
fs/ext2/file.c | 5 +
fs/ext4/file.c | 3 -
fs/f2fs/file.c | 3 -
fs/fcntl.c | 15 +++
fs/fuse/file.c | 8 +
fs/gfs2/file.c | 3 -
fs/hugetlbfs/inode.c | 3 -
fs/kernfs/file.c | 3 -
fs/ncpfs/mmap.c | 3 -
fs/ncpfs/ncp_fs.h | 2
fs/nfs/file.c | 5 +
fs/nfs/internal.h | 2
fs/nilfs2/file.c | 3 -
fs/ocfs2/mmap.c | 3 -
fs/ocfs2/mmap.h | 3 -
fs/open.c | 6 +
fs/orangefs/file.c | 5 +
fs/proc/inode.c | 7 +
fs/proc/vmcore.c | 6 +
fs/ramfs/file-nommu.c | 6 +
fs/read_write.c | 3 +
fs/romfs/mmap-nommu.c | 3 -
fs/ubifs/file.c | 5 +
fs/xfs/libxfs/xfs_bmap.c | 5 +
fs/xfs/xfs_bmap_util.c | 3 +
fs/xfs/xfs_file.c | 114 +++++++++++++++++++-
fs/xfs/xfs_inode.h | 1
fs/xfs/xfs_ioctl.c | 6 +
fs/xfs/xfs_super.c | 1
include/drm/drm_gem.h | 3 -
include/drm/drm_gem_cma_helper.h | 3 -
include/drm/drm_legacy.h | 3 -
include/linux/fs.h | 21 ++--
include/linux/mm.h | 2
include/linux/mman.h | 46 ++++++++
include/linux/syscalls.h | 3 +
include/misc/cxl.h | 3 -
include/uapi/asm-generic/mman.h | 1
include/uapi/linux/fcntl.h | 5 +
ipc/shm.c | 5 +
kernel/events/core.c | 3 -
kernel/kcov.c | 3 -
kernel/relay.c | 3 -
mm/filemap.c | 19 ++-
mm/mmap.c | 56 +++++++++-
mm/nommu.c | 4 -
mm/shmem.c | 3 -
net/socket.c | 6 +
security/selinux/selinuxfs.c | 6 +
sound/core/compress_offload.c | 3 -
sound/core/hwdep.c | 3 -
sound/core/info.c | 3 -
sound/core/init.c | 3 -
sound/core/oss/pcm_oss.c | 3 -
sound/core/pcm_native.c | 3 -
sound/oss/soundcard.c | 3 -
sound/oss/swarm_cs4297a.c | 3 -
virt/kvm/kvm_main.c | 3 -
177 files changed, 689 insertions(+), 234 deletions(-)
next reply other threads:[~2017-08-23 23:48 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-23 23:48 Dan Williams [this message]
[not found] ` <150353211413.5039.5228914877418362329.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-23 23:48 ` [PATCH v6 1/5] vfs: add flags parameter to ->mmap() in 'struct file_operations' Dan Williams
[not found] ` <150353211985.5039.4333061601382775843.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-24 16:58 ` Christoph Hellwig
2017-08-24 17:42 ` Dan Williams
2017-08-23 23:48 ` [PATCH v6 2/5] fs, xfs: introduce S_IOMAP_SEALED Dan Williams
[not found] ` <150353212577.5039.14069456126848863439.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-24 16:13 ` Christoph Hellwig
2017-08-25 6:00 ` Dan Williams
2017-08-25 19:44 ` Dan Williams
2017-08-23 23:48 ` [PATCH v6 4/5] fs, xfs: introduce MAP_DIRECT for creating block-map-atomic file ranges Dan Williams
[not found] ` <150353213655.5039.7662200155640827407.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-24 16:11 ` Christoph Hellwig
[not found] ` <20170824161152.GB27591-jcswGhMUV9g@public.gmane.org>
2017-08-24 16:31 ` Dan Williams
2017-08-24 16:39 ` Christoph Hellwig
[not found] ` <20170824163925.GA28503-jcswGhMUV9g@public.gmane.org>
2017-08-24 20:26 ` Dan Williams
2017-08-23 23:49 ` [PATCH v6 5/5] fs, fcntl: add F_MAP_DIRECT Dan Williams
2017-08-23 23:48 ` [PATCH v6 3/5] mm: introduce mmap3 for safely defining new mmap flags Dan Williams
[not found] ` <150353213097.5039.6729469069608762658.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-24 13:04 ` Jan Kara
2017-08-24 16:55 ` Christoph Hellwig
[not found] ` <20170824165546.GA3121-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2017-08-24 17:36 ` Dan Williams
[not found] ` <CAPcyv4iN0QpUSgOUvisnNQsiV1Pp=4dh7CwAV8FFj=_rFU=aug-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-08-25 13:00 ` Christoph Hellwig
2017-08-25 15:58 ` Kirill A. Shutemov
2017-08-25 16:02 ` Christoph Hellwig
2017-08-25 16:16 ` Kirill A. Shutemov
[not found] ` <20170825161607.6v6beg4zjktllt2z-sVvlyX1904swdBt8bTSxpkEMvNT87kid@public.gmane.org>
2017-08-25 16:19 ` Helge Deller
2017-08-25 16:56 ` Kirill A. Shutemov
2017-08-25 20:24 ` Dan Williams
[not found] ` <CAPcyv4jeZc8P+E0aHNChzy-wfNpOx3GehKck1nXqJ1b9JdydFA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-08-26 7:40 ` Helge Deller
2017-08-26 15:15 ` Dan Williams
[not found] ` <CAPcyv4ic0zxQzWEipZ=1LpDC8VnmphGzVSYmrFcjOAgX7esfUw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-08-26 19:50 ` Helge Deller
2017-08-26 22:46 ` Dan Williams
2017-08-26 23:56 ` Kirill A. Shutemov
2017-08-24 16:08 ` [PATCH v6 0/5] MAP_DIRECT and block-map-atomic files Christoph Hellwig
2017-08-24 16:25 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=150353211413.5039.5228914877418362329.stgit@dwillia2-desk3.amr.corp.intel.com \
--to=dan.j.williams-ral2jqcrhueavxtiumwx3w@public.gmane.org \
--cc=airlied-cv59FeDIM0c@public.gmane.org \
--cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
--cc=arnd-r2nGTMty4D4@public.gmane.org \
--cc=daniel.vetter-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=darrick.wong-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
--cc=david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org \
--cc=dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org \
--cc=hch-jcswGhMUV9g@public.gmane.org \
--cc=jack-AlSwsSmVLrQ@public.gmane.org \
--cc=julia.lawall-L2FTfq7BK8M@public.gmane.org \
--cc=linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
--cc=linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org \
--cc=linux-xfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
--cc=tiwai-IBi9RG/b67k@public.gmane.org \
--cc=viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).