From: Ira Weiny <ira.weiny@intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Michal Hocko <mhocko@suse.com>, Theodore Ts'o <tytso@mit.edu>,
linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org,
John Hubbard <jhubbard@nvidia.com>,
Dave Chinner <david@fromorbit.com>,
linux-kernel@vger.kernel.org,
Matthew Wilcox <willy@infradead.org>,
linux-xfs@vger.kernel.org, linux-mm@kvack.org,
linux-fsdevel@vger.kernel.org, Jan Kara <jack@suse.cz>,
Andrew Morton <akpm@linux-foundation.org>,
linux-ext4@vger.kernel.org
Subject: Re: [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -)
Date: Mon, 19 Aug 2019 14:53:23 -0700 [thread overview]
Message-ID: <20190819215322.GA2839@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <20190819123841.GC5058@ziepe.ca>
On Mon, Aug 19, 2019 at 09:38:41AM -0300, Jason Gunthorpe wrote:
> On Mon, Aug 19, 2019 at 07:24:09PM +1000, Dave Chinner wrote:
>
> > So that leaves just the normal close() syscall exit case, where the
> > application has full control of the order in which resources are
> > released. We've already established that we can block in this
> > context. Blocking in an interruptible state will allow fatal signal
> > delivery to wake us, and then we fall into the
> > fatal_signal_pending() case if we get a SIGKILL while blocking.
>
> The major problem with RDMA is that it doesn't always wait on close() for the
> MR holding the page pins to be destoyed. This is done to avoid a
> deadlock of the form:
>
> uverbs_destroy_ufile_hw()
> mutex_lock()
> [..]
> mmput()
> exit_mmap()
> remove_vma()
> fput();
> file_operations->release()
> ib_uverbs_close()
> uverbs_destroy_ufile_hw()
> mutex_lock() <-- Deadlock
>
> But, as I said to Ira earlier, I wonder if this is now impossible on
> modern kernels and we can switch to making the whole thing
> synchronous. That would resolve RDMA's main problem with this.
I'm still looking into this... but my bigger concern is that the RDMA FD can
be passed to other processes via SCM_RIGHTS. Which means the process holding
the pin may _not_ be the one with the open file and layout lease...
Ira
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Ira Weiny <ira.weiny@intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Dave Chinner <david@fromorbit.com>, Jan Kara <jack@suse.cz>,
Andrew Morton <akpm@linux-foundation.org>,
Dan Williams <dan.j.williams@intel.com>,
Matthew Wilcox <willy@infradead.org>,
Theodore Ts'o <tytso@mit.edu>, John Hubbard <jhubbard@nvidia.com>,
Michal Hocko <mhocko@suse.com>,
linux-xfs@vger.kernel.org, linux-rdma@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-nvdimm@lists.01.org, linux-ext4@vger.kernel.org,
linux-mm@kvack.org
Subject: Re: [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-)
Date: Mon, 19 Aug 2019 14:53:23 -0700 [thread overview]
Message-ID: <20190819215322.GA2839@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <20190819123841.GC5058@ziepe.ca>
On Mon, Aug 19, 2019 at 09:38:41AM -0300, Jason Gunthorpe wrote:
> On Mon, Aug 19, 2019 at 07:24:09PM +1000, Dave Chinner wrote:
>
> > So that leaves just the normal close() syscall exit case, where the
> > application has full control of the order in which resources are
> > released. We've already established that we can block in this
> > context. Blocking in an interruptible state will allow fatal signal
> > delivery to wake us, and then we fall into the
> > fatal_signal_pending() case if we get a SIGKILL while blocking.
>
> The major problem with RDMA is that it doesn't always wait on close() for the
> MR holding the page pins to be destoyed. This is done to avoid a
> deadlock of the form:
>
> uverbs_destroy_ufile_hw()
> mutex_lock()
> [..]
> mmput()
> exit_mmap()
> remove_vma()
> fput();
> file_operations->release()
> ib_uverbs_close()
> uverbs_destroy_ufile_hw()
> mutex_lock() <-- Deadlock
>
> But, as I said to Ira earlier, I wonder if this is now impossible on
> modern kernels and we can switch to making the whole thing
> synchronous. That would resolve RDMA's main problem with this.
I'm still looking into this... but my bigger concern is that the RDMA FD can
be passed to other processes via SCM_RIGHTS. Which means the process holding
the pin may _not_ be the one with the open file and layout lease...
Ira
next prev parent reply other threads:[~2019-08-19 21:54 UTC|newest]
Thread overview: 201+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-09 22:58 [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 01/19] fs/locks: Export F_LAYOUT lease to user space ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 23:52 ` Dave Chinner
2019-08-09 23:52 ` Dave Chinner
2019-08-12 17:36 ` Ira Weiny
2019-08-12 17:36 ` Ira Weiny
2019-08-14 8:05 ` Dave Chinner
2019-08-14 8:05 ` Dave Chinner
2019-08-14 11:21 ` Jeff Layton
2019-08-14 11:21 ` Jeff Layton
2019-08-14 11:21 ` Jeff Layton
2019-08-14 11:38 ` Dave Chinner
2019-08-14 11:38 ` Dave Chinner
2019-08-09 22:58 ` [RFC PATCH v2 02/19] fs/locks: Add Exclusive flag to user Layout lease ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-14 14:15 ` Jeff Layton
2019-08-14 14:15 ` Jeff Layton
2019-08-14 14:15 ` Jeff Layton
2019-08-14 21:56 ` Dave Chinner
2019-08-26 10:41 ` Jeff Layton
2019-08-26 10:41 ` Jeff Layton
2019-08-26 10:41 ` Jeff Layton
2019-08-29 23:34 ` Ira Weiny
2019-08-29 23:34 ` Ira Weiny
2019-09-04 12:52 ` Jeff Layton
2019-09-04 12:52 ` Jeff Layton
2019-09-04 23:12 ` John Hubbard
2019-09-04 23:12 ` John Hubbard
2019-08-09 22:58 ` [RFC PATCH v2 03/19] mm/gup: Pass flags down to __gup_device_huge* calls ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 04/19] mm/gup: Ensure F_LAYOUT lease is held prior to GUP'ing pages ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 05/19] fs/ext4: Teach ext4 to break layout leases ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 06/19] fs/ext4: Teach dax_layout_busy_page() to operate on a sub-range ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-23 15:18 ` Vivek Goyal
2019-08-23 15:18 ` Vivek Goyal
2019-08-29 18:52 ` Ira Weiny
2019-08-29 18:52 ` Ira Weiny
2019-08-09 22:58 ` [RFC PATCH v2 07/19] fs/xfs: Teach xfs to use new dax_layout_busy_page() ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 23:30 ` Dave Chinner
2019-08-09 23:30 ` Dave Chinner
2019-08-12 18:05 ` Ira Weiny
2019-08-12 18:05 ` Ira Weiny
2019-08-14 8:04 ` Dave Chinner
2019-08-09 22:58 ` [RFC PATCH v2 08/19] fs/xfs: Fail truncate if page lease can't be broken ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 23:22 ` Dave Chinner
2019-08-09 23:22 ` Dave Chinner
2019-08-12 18:08 ` Ira Weiny
2019-08-09 22:58 ` [RFC PATCH v2 09/19] mm/gup: Introduce vaddr_pin structure ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-10 0:06 ` John Hubbard
2019-08-10 0:06 ` John Hubbard
2019-08-10 0:06 ` John Hubbard
2019-08-09 22:58 ` [RFC PATCH v2 10/19] mm/gup: Pass a NULL vaddr_pin through GUP fast ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-10 0:06 ` John Hubbard
2019-08-10 0:06 ` John Hubbard
2019-08-10 0:06 ` John Hubbard
2019-08-09 22:58 ` [RFC PATCH v2 11/19] mm/gup: Pass follow_page_context further down the call stack ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-10 0:18 ` John Hubbard
2019-08-10 0:18 ` John Hubbard
2019-08-10 0:18 ` John Hubbard
2019-08-12 19:01 ` Ira Weiny
2019-08-12 19:01 ` Ira Weiny
2019-08-09 22:58 ` [RFC PATCH v2 12/19] mm/gup: Prep put_user_pages() to take an vaddr_pin struct ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-10 0:30 ` John Hubbard
2019-08-10 0:30 ` John Hubbard
2019-08-10 0:30 ` John Hubbard
2019-08-12 20:46 ` Ira Weiny
2019-08-09 22:58 ` [RFC PATCH v2 13/19] {mm,file}: Add file_pins objects ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 14/19] fs/locks: Associate file pins while performing GUP ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 15/19] mm/gup: Introduce vaddr_pin_pages() ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-10 0:09 ` John Hubbard
2019-08-10 0:09 ` John Hubbard
2019-08-10 0:09 ` John Hubbard
2019-08-12 21:00 ` Ira Weiny
2019-08-12 21:00 ` Ira Weiny
2019-08-12 21:20 ` John Hubbard
2019-08-12 21:20 ` John Hubbard
2019-08-12 21:20 ` John Hubbard
2019-08-11 23:07 ` John Hubbard
2019-08-11 23:07 ` John Hubbard
2019-08-11 23:07 ` John Hubbard
2019-08-12 21:01 ` Ira Weiny
2019-08-12 12:28 ` Jason Gunthorpe
2019-08-12 21:48 ` Ira Weiny
2019-08-12 21:48 ` Ira Weiny
2019-08-13 11:47 ` Jason Gunthorpe
2019-08-13 17:46 ` Ira Weiny
2019-08-13 17:46 ` Ira Weiny
2019-08-13 17:56 ` John Hubbard
2019-08-13 17:56 ` John Hubbard
2019-08-13 17:56 ` John Hubbard
2019-08-09 22:58 ` [RFC PATCH v2 16/19] RDMA/uverbs: Add back pointer to system file object ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-12 13:00 ` Jason Gunthorpe
2019-08-12 17:28 ` Ira Weiny
2019-08-12 17:56 ` Jason Gunthorpe
2019-08-12 21:15 ` Ira Weiny
2019-08-12 21:15 ` Ira Weiny
2019-08-13 11:48 ` Jason Gunthorpe
2019-08-13 17:41 ` Ira Weiny
2019-08-13 17:41 ` Ira Weiny
2019-08-13 18:00 ` Jason Gunthorpe
2019-08-13 20:38 ` Ira Weiny
2019-08-13 20:38 ` Ira Weiny
2019-08-14 12:23 ` Jason Gunthorpe
2019-08-14 17:50 ` Ira Weiny
2019-08-14 18:15 ` Jason Gunthorpe
2019-09-04 22:25 ` Ira Weiny
2019-09-11 8:19 ` Jason Gunthorpe
2019-08-09 22:58 ` [RFC PATCH v2 17/19] RDMA/umem: Convert to vaddr_[pin|unpin]* operations ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 18/19] {mm,procfs}: Add display file_pins proc ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-09 22:58 ` [RFC PATCH v2 19/19] mm/gup: Remove FOLL_LONGTERM DAX exclusion ira.weiny
2019-08-09 22:58 ` ira.weiny
2019-08-14 10:17 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Jan Kara
2019-08-14 18:08 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-14 18:08 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-15 13:05 ` Jan Kara
2019-08-16 19:05 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-16 19:05 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-16 23:20 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-16 23:20 ` Ira Weiny
2019-08-19 6:36 ` Jan Kara
2019-08-19 6:36 ` Jan Kara
2019-08-17 2:26 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-17 2:26 ` Dave Chinner
2019-08-19 6:34 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Jan Kara
2019-08-19 6:34 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Jan Kara
2019-08-19 9:24 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-19 9:24 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-19 12:38 ` Jason Gunthorpe
2019-08-19 21:53 ` Ira Weiny [this message]
2019-08-19 21:53 ` Ira Weiny
2019-08-20 1:12 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-20 1:12 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-20 11:55 ` Jason Gunthorpe
2019-08-21 18:02 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-21 18:02 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-21 18:13 ` Jason Gunthorpe
2019-08-21 18:22 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) John Hubbard
2019-08-21 18:22 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) John Hubbard
2019-08-21 18:57 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-21 18:57 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-21 19:06 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-21 19:06 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-21 19:48 ` Jason Gunthorpe
2019-08-21 20:44 ` Ira Weiny
2019-08-21 23:49 ` Jason Gunthorpe
2019-08-23 3:23 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-23 3:23 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-23 12:04 ` Jason Gunthorpe
2019-08-24 0:11 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-24 0:11 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-24 5:08 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-24 5:08 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-26 5:55 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-26 5:55 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-29 2:02 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-29 2:02 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-29 3:27 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) John Hubbard
2019-08-29 3:27 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) John Hubbard
2019-08-29 16:16 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-29 16:16 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-09-02 22:26 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-09-02 22:26 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-09-04 16:54 ` Ira Weiny
2019-08-25 19:39 ` Jason Gunthorpe
2019-08-24 4:49 ` Ira Weiny
2019-08-24 4:49 ` Ira Weiny
2019-08-25 19:40 ` Jason Gunthorpe
2019-08-23 0:59 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-23 0:59 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-23 17:15 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-23 17:15 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
2019-08-24 0:18 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-24 0:18 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-20 0:05 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) John Hubbard
2019-08-20 0:05 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) John Hubbard
2019-08-20 1:20 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Dave Chinner
2019-08-20 1:20 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Dave Chinner
2019-08-20 3:09 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) John Hubbard
2019-08-20 3:09 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) John Hubbard
2019-08-20 3:36 ` Dave Chinner
2019-08-21 18:43 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) John Hubbard
2019-08-21 18:43 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) John Hubbard
2019-08-21 19:09 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ; -) Ira Weiny
2019-08-21 19:09 ` [RFC PATCH v2 00/19] RDMA/FS DAX truncate proposal V1,000,002 ;-) Ira Weiny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190819215322.GA2839@iweiny-DESK2.sc.intel.com \
--to=ira.weiny@intel.com \
--cc=akpm@linux-foundation.org \
--cc=david@fromorbit.com \
--cc=jack@suse.cz \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=mhocko@suse.com \
--cc=tytso@mit.edu \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.