Re: [PATCH v9 0/6] MAP_DIRECT for DAX userspace flush

[Christoph Hellwig <hch@lst.de>]

On Fri, Oct 13, 2017 at 11:31:45AM -0600, Jason Gunthorpe wrote:
> I don't think that really represents how lots of apps actually use
> RDMA.
> 
> RDMA is often buried down in the software stack (eg in a MPI), and by
> the time a mapping gets used for RDMA transfer the link between the
> FD, mmap and the MR is totally opaque.
> 
> Having a MR specific notification means the low level RDMA libraries
> have a chance to deal with everything for the app.
> 
> Eg consider a HPC app using MPI that uses some DAX aware library to
> get DAX backed mmap's. It then passes memory in those mmaps to the
> MPI library to do transfers. The MPI creates the MR on demand.
> 

I suspect one of the more interesting use cases might be a file server,
for which that's not the case.  But otherwise I agree with the above,
and also thing that notifying the MR handle is the only way to go for
another very important reason:  fencing.  What if the application/library
does not react on the notification?  With a per-MR notification we
can unregister the MR in kernel space and have a rock solid fencing
mechanism.  And that is the most important bit here.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>