From: Dave Chinner <david@fromorbit.com>
To: Josef Bacik <josef@toxicpanda.com>
Cc: kernel-team@fb.com, hannes@cmpxchg.org,
linux-kernel@vger.kernel.org, tj@kernel.org,
akpm@linux-foundation.org, linux-fsdevel@vger.kernel.org,
linux-btrfs@vger.kernel.org, riel@fb.com, linux-mm@kvack.org
Subject: Re: [PATCH 7/7] btrfs: drop mmap_sem in mkwrite for btrfs
Date: Tue, 23 Oct 2018 08:31:33 +1100 [thread overview]
Message-ID: <20181022213133.GB6311@dastard> (raw)
In-Reply-To: <20181022175652.ase7u23uzizqtlao@destiny>
On Mon, Oct 22, 2018 at 01:56:54PM -0400, Josef Bacik wrote:
> On Fri, Oct 19, 2018 at 02:48:47PM +1100, Dave Chinner wrote:
> > On Thu, Oct 18, 2018 at 04:23:18PM -0400, Josef Bacik wrote:
> > > ->page_mkwrite is extremely expensive in btrfs. We have to reserve
> > > space, which can take 6 lifetimes, and we could possibly have to wait on
> > > writeback on the page, another several lifetimes. To avoid this simply
> > > drop the mmap_sem if we didn't have the cached page and do all of our
> > > work and return the appropriate retry error. If we have the cached page
> > > we know we did all the right things to set this page up and we can just
> > > carry on.
> > >
> > > Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> > > ---
> > > fs/btrfs/inode.c | 41 +++++++++++++++++++++++++++++++++++++++--
> > > include/linux/mm.h | 14 ++++++++++++++
> > > mm/filemap.c | 3 ++-
> > > 3 files changed, 55 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
> > > index 3ea5339603cf..6b723d29bc0c 100644
> > > --- a/fs/btrfs/inode.c
> > > +++ b/fs/btrfs/inode.c
> > > @@ -8809,7 +8809,9 @@ static void btrfs_invalidatepage(struct page *page, unsigned int offset,
> > > vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
> > > {
> > > struct page *page = vmf->page;
> > > - struct inode *inode = file_inode(vmf->vma->vm_file);
> > > + struct file *file = vmf->vma->vm_file, *fpin;
> > > + struct mm_struct *mm = vmf->vma->vm_mm;
> > > + struct inode *inode = file_inode(file);
> > > struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
> > > struct extent_io_tree *io_tree = &BTRFS_I(inode)->io_tree;
> > > struct btrfs_ordered_extent *ordered;
> > > @@ -8828,6 +8830,29 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)
> > >
> > > reserved_space = PAGE_SIZE;
> > >
> > > + /*
> > > + * We have our cached page from a previous mkwrite, check it to make
> > > + * sure it's still dirty and our file size matches when we ran mkwrite
> > > + * the last time. If everything is OK then return VM_FAULT_LOCKED,
> > > + * otherwise do the mkwrite again.
> > > + */
> > > + if (vmf->flags & FAULT_FLAG_USED_CACHED) {
> > > + lock_page(page);
> > > + if (vmf->cached_size == i_size_read(inode) &&
> > > + PageDirty(page))
> > > + return VM_FAULT_LOCKED;
> > > + unlock_page(page);
> > > + }
> >
> > What does the file size have to do with whether we can use the
> > initialised page or not? The file can be extended by other
> > data operations (like write()) while a page fault is in progress,
> > so I'm not sure how or why this check makes any sense.
> >
> > I also don't see anything btrfs specific here, so....
> >
>
> First thanks for the review, I've gone through and addressed everything you
> mentioned, however this one is subtle.
>
> The problem is the vmf->vma->vm_file access. Once we drop the mmap_sem we can
> no longer safely go into vmf->vma, so I'd have to fix all the page_mkwrite()'s
> to not touch vma, and add a vmf->fpin instead to mess with.
Adding a vmf->file pointer seems pretty easy - making /everything/
use it instead of just special casing page_mkwrite also seems like
it would be a good idea - set it up in your new init function and
it's done. Pinning and unpinning could be done unconditionally, too
- that doesn't look expensive - and it would pull a lot of the
complexity out of the patchset for the cases where unlocking the
mmap_sem is done....
> Plus I didn't want
> to miss some subtlety in other fs's page_mkwrite()'s and inavertedly break them.
> If I break btrfs I can fix it, I'm not as good with xfs.
Sure, but if you make it a generic helper then you don't have to
worry about that. i.e.
generic_page_mkwrite_nommapsem(vmf, page_mkwrite_cb)
{
/* use cached page if valid */
/* unlock mmap_sem */
/* do filesystem page_mkwrite callback */
ret = page_mkwrite_cb(vmf);
/* handle page caching */
/* return retry fault indicator */
}
and that allows all filesystems to easily opt in to dropping the
mmap_sem and you don't have to worry about it too much. It means
filesystems with dax fault paths (like XFS) can convert the non-dax
paths that call immediately, and worry about the more tricky stuff
later...
> vmf->fpin and vmf->mm, and then fix up all the mkwrite()'s to use that instead
Ignoring DAX (because I haven't checked), what filesystem
page_mkwrite implementation requires access to the mm_struct? They
should all just be doing file operations on a page, not anything to
do with the way the page is mapped into the task memory....
> of vmf->vma. I think that's the only things we care about so it wouldn't be
> hard. Does that sound reasonable to you?
I think a little bit of tweaking, and it will be easy to for
everyone to opt in to this behaviour....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2018-10-22 21:32 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-18 20:23 [PATCH 0/7][V3] drop the mmap_sem when doing IO in the fault path Josef Bacik
2018-10-18 20:23 ` [PATCH 1/7] mm: infrastructure for page fault page caching Josef Bacik
2018-10-18 20:23 ` [PATCH 2/7] mm: drop mmap_sem for page cache read IO submission Josef Bacik
2018-10-19 3:14 ` Dave Chinner
2018-10-18 20:23 ` [PATCH 3/7] mm: drop the mmap_sem in all read fault cases Josef Bacik
2018-10-19 3:21 ` Dave Chinner
2018-10-18 20:23 ` [PATCH 4/7] mm: use the cached page for filemap_fault Josef Bacik
2018-10-19 3:27 ` Dave Chinner
2018-10-18 20:23 ` [PATCH 5/7] mm: add a flag to indicate we used a cached page Josef Bacik
2018-10-19 3:34 ` Dave Chinner
2018-10-18 20:23 ` [PATCH 6/7] mm: allow ->page_mkwrite to do retries Josef Bacik
2018-10-19 3:36 ` Dave Chinner
2018-10-18 20:23 ` [PATCH 7/7] btrfs: drop mmap_sem in mkwrite for btrfs Josef Bacik
2018-10-19 3:48 ` Dave Chinner
2018-10-22 17:56 ` Josef Bacik
2018-10-22 21:31 ` Dave Chinner [this message]
2018-10-25 13:22 ` Jan Kara
2018-10-25 13:58 ` Josef Bacik
2018-10-26 9:47 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181022213133.GB6311@dastard \
--to=david@fromorbit.com \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=riel@fb.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).