From: Dave Chinner <david@fromorbit.com>
To: Benjamin LaHaise <bcrl@kvack.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
linux-aio@kvack.org,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Linux API <linux-api@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 07/13] aio: enabled thread based async fsync
Date: Thu, 21 Jan 2016 08:45:46 +1100 [thread overview]
Message-ID: <20160120214546.GX6033@dastard> (raw)
In-Reply-To: <20160120204449.GC12249@kvack.org>
On Wed, Jan 20, 2016 at 03:44:49PM -0500, Benjamin LaHaise wrote:
> On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote:
> > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner <david@fromorbit.com> wrote:
> > >>
> > >> Are there other users outside of Solace? It would be good to get comments..
> > >
> > > I know of quite a few storage/db products that use AIO. The most
> > > recent high profile project that have been reporting issues with AIO
> > > on XFS is http://www.scylladb.com/. That project is architected
> > > around non-blocking AIO for scalability reasons...
> >
> > I was more wondering about the new interfaces, making sure that the
> > feature set actually matches what people want to do..
>
> I suspect this will be an ongoing learning exercise as people start to use
> the new functionality and find gaps in terms of what is needed. Certainly
> there is a bunch of stuff we need to add to cover the cases where disk i/o
> is required. getdents() is one example, but the ABI issues we have with it
> are somewhat more complicated given the history associated with that
> interface.
>
> > That said, I also agree that it would be interesting to hear what the
> > performance impact is for existing performance-sensitive users. Could
> > we make that "aio_may_use_threads()" case be unconditional, making
> > things simpler?
>
> Making it unconditional is a goal, but some work is required before that
> can be the case. The O_DIRECT issue is one such matter -- it requires some
> changes to the filesystems to ensure that they adhere to the non-blocking
> nature of the new interface (ie taking i_mutex is a Bad Thing that users
> really do not want to be exposed to; if taking it blocks, the code should
> punt to a helper thread).
Filesystems *must take locks* in the IO path. We have to serialise
against truncate and other operations at some point in the IO path
(e.g. block mapping vs concurrent allocation and/or removal), and
that can only be done sanely with sleeping locks. There is no way
of knowing in advance if we are going to block, and so either we
always use threads for IO submission or we accept that occasionally
the AIO submission will block.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com>
To: Benjamin LaHaise <bcrl@kvack.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
linux-aio@kvack.org,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Linux API <linux-api@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 07/13] aio: enabled thread based async fsync
Date: Thu, 21 Jan 2016 08:45:46 +1100 [thread overview]
Message-ID: <20160120214546.GX6033@dastard> (raw)
In-Reply-To: <20160120204449.GC12249@kvack.org>
On Wed, Jan 20, 2016 at 03:44:49PM -0500, Benjamin LaHaise wrote:
> On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote:
> > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner <david@fromorbit.com> wrote:
> > >>
> > >> Are there other users outside of Solace? It would be good to get comments..
> > >
> > > I know of quite a few storage/db products that use AIO. The most
> > > recent high profile project that have been reporting issues with AIO
> > > on XFS is http://www.scylladb.com/. That project is architected
> > > around non-blocking AIO for scalability reasons...
> >
> > I was more wondering about the new interfaces, making sure that the
> > feature set actually matches what people want to do..
>
> I suspect this will be an ongoing learning exercise as people start to use
> the new functionality and find gaps in terms of what is needed. Certainly
> there is a bunch of stuff we need to add to cover the cases where disk i/o
> is required. getdents() is one example, but the ABI issues we have with it
> are somewhat more complicated given the history associated with that
> interface.
>
> > That said, I also agree that it would be interesting to hear what the
> > performance impact is for existing performance-sensitive users. Could
> > we make that "aio_may_use_threads()" case be unconditional, making
> > things simpler?
>
> Making it unconditional is a goal, but some work is required before that
> can be the case. The O_DIRECT issue is one such matter -- it requires some
> changes to the filesystems to ensure that they adhere to the non-blocking
> nature of the new interface (ie taking i_mutex is a Bad Thing that users
> really do not want to be exposed to; if taking it blocks, the code should
> punt to a helper thread).
Filesystems *must take locks* in the IO path. We have to serialise
against truncate and other operations at some point in the IO path
(e.g. block mapping vs concurrent allocation and/or removal), and
that can only be done sanely with sleeping locks. There is no way
of knowing in advance if we are going to block, and so either we
always use threads for IO submission or we accept that occasionally
the AIO submission will block.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-01-20 21:46 UTC|newest]
Thread overview: 133+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-11 22:06 [PATCH 00/13] aio: thread (work queue) based aio and new aio functionality Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` [PATCH 01/13] signals: distinguish signals sent due to i/o via io_send_sig() Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` [PATCH 02/13] aio: add aio_get_mm() helper Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` [PATCH 03/13] aio: for async operations, make the iter argument persistent Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:06 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 04/13] signals: add and use aio_get_task() to direct signals sent via io_send_sig() Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 05/13] fs: make do_loop_readv_writev() non-static Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 06/13] aio: add queue_work() based threaded aio support Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 07/13] aio: enabled thread based async fsync Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-12 1:11 ` Dave Chinner
2016-01-12 1:11 ` Dave Chinner
2016-01-12 1:20 ` Linus Torvalds
2016-01-12 1:20 ` Linus Torvalds
2016-01-12 2:25 ` Dave Chinner
2016-01-12 2:25 ` Dave Chinner
2016-01-12 2:25 ` Dave Chinner
2016-01-12 2:38 ` Linus Torvalds
2016-01-12 2:38 ` Linus Torvalds
2016-01-12 3:37 ` Dave Chinner
2016-01-12 3:37 ` Dave Chinner
2016-01-12 4:03 ` Linus Torvalds
2016-01-12 4:03 ` Linus Torvalds
2016-01-12 4:48 ` Linus Torvalds
2016-01-12 4:48 ` Linus Torvalds
2016-01-12 22:50 ` Benjamin LaHaise
2016-01-12 22:50 ` Benjamin LaHaise
2016-01-12 22:50 ` Benjamin LaHaise
2016-01-15 20:21 ` Benjamin LaHaise
2016-01-15 20:21 ` Benjamin LaHaise
2016-01-15 20:21 ` Benjamin LaHaise
2016-01-20 3:59 ` Linus Torvalds
2016-01-20 3:59 ` Linus Torvalds
2016-01-20 3:59 ` Linus Torvalds
2016-01-20 5:02 ` Theodore Ts'o
2016-01-20 5:02 ` Theodore Ts'o
2016-01-20 5:02 ` Theodore Ts'o
2016-01-20 19:59 ` Dave Chinner
2016-01-20 19:59 ` Dave Chinner
2016-01-20 19:59 ` Dave Chinner
2016-01-20 20:29 ` Linus Torvalds
2016-01-20 20:29 ` Linus Torvalds
2016-01-20 20:44 ` Benjamin LaHaise
2016-01-20 20:44 ` Benjamin LaHaise
2016-01-20 20:44 ` Benjamin LaHaise
2016-01-20 21:45 ` Dave Chinner [this message]
2016-01-20 21:45 ` Dave Chinner
2016-01-20 21:56 ` Benjamin LaHaise
2016-01-20 21:56 ` Benjamin LaHaise
2016-01-20 21:56 ` Benjamin LaHaise
2016-01-23 4:24 ` Dave Chinner
2016-01-23 4:24 ` Dave Chinner
2016-01-23 4:50 ` Benjamin LaHaise
2016-01-23 4:50 ` Benjamin LaHaise
2016-01-23 4:50 ` Benjamin LaHaise
2016-01-23 22:22 ` Dave Chinner
2016-01-23 22:22 ` Dave Chinner
2016-01-23 22:22 ` Dave Chinner
2016-01-20 23:07 ` Linus Torvalds
2016-01-23 4:39 ` Dave Chinner
2016-01-23 4:39 ` Dave Chinner
2016-01-23 4:39 ` Dave Chinner
2016-03-14 17:17 ` aio openat " Benjamin LaHaise
2016-03-14 17:17 ` Benjamin LaHaise
2016-03-20 1:20 ` Linus Torvalds
2016-03-20 1:20 ` Linus Torvalds
2016-03-20 1:26 ` Al Viro
2016-03-20 1:26 ` Al Viro
2016-03-20 1:26 ` Al Viro
2016-03-20 1:45 ` Linus Torvalds
2016-03-20 1:45 ` Linus Torvalds
2016-03-20 1:45 ` Linus Torvalds
2016-03-20 1:55 ` Al Viro
2016-03-20 1:55 ` Al Viro
2016-03-20 2:03 ` Linus Torvalds
2016-03-20 2:03 ` Linus Torvalds
2016-03-20 2:03 ` Linus Torvalds
2016-01-20 21:57 ` Dave Chinner
2016-01-20 21:57 ` Dave Chinner
2016-01-20 21:57 ` Dave Chinner
2016-01-22 15:41 ` Andres Freund
2016-01-22 15:41 ` Andres Freund
2016-01-12 22:59 ` Andy Lutomirski
2016-01-12 22:59 ` Andy Lutomirski
2016-01-12 22:59 ` Andy Lutomirski
2016-01-14 9:19 ` Paolo Bonzini
2016-01-14 9:19 ` Paolo Bonzini
2016-01-14 9:19 ` Paolo Bonzini
2016-01-12 1:30 ` Benjamin LaHaise
2016-01-12 1:30 ` Benjamin LaHaise
2016-01-12 1:30 ` Benjamin LaHaise
2016-01-22 15:31 ` Andres Freund
2016-01-22 15:31 ` Andres Freund
2016-01-22 15:31 ` Andres Freund
2016-01-11 22:07 ` [PATCH 08/13] aio: add support for aio poll via aio thread helper Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 09/13] aio: add support for async openat() Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-12 0:22 ` Linus Torvalds
2016-01-12 0:22 ` Linus Torvalds
2016-01-12 1:17 ` Benjamin LaHaise
2016-01-12 1:17 ` Benjamin LaHaise
2016-01-12 1:17 ` Benjamin LaHaise
2016-01-12 1:45 ` Chris Mason
2016-01-12 1:45 ` Chris Mason
2016-01-12 1:45 ` Chris Mason
2016-01-12 9:53 ` Ingo Molnar
2016-01-12 9:53 ` Ingo Molnar
2016-01-12 9:53 ` Ingo Molnar
2016-01-11 22:07 ` [PATCH 10/13] aio: add async unlinkat functionality Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 11/13] mm: enable __do_page_cache_readahead() to include present pages Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:07 ` [PATCH 12/13] aio: add support for aio readahead Benjamin LaHaise
2016-01-11 22:07 ` Benjamin LaHaise
2016-01-11 22:08 ` [PATCH 13/13] aio: add support for aio renameat operation Benjamin LaHaise
2016-01-11 22:08 ` Benjamin LaHaise
2016-01-11 22:08 ` Benjamin LaHaise
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160120214546.GX6033@dastard \
--to=david@fromorbit.com \
--cc=akpm@linux-foundation.org \
--cc=bcrl@kvack.org \
--cc=linux-aio@kvack.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.