From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758277AbcATUoy (ORCPT ); Wed, 20 Jan 2016 15:44:54 -0500 Received: from kanga.kvack.org ([205.233.56.17]:55107 "EHLO kanga.kvack.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750781AbcATUow (ORCPT ); Wed, 20 Jan 2016 15:44:52 -0500 Date: Wed, 20 Jan 2016 15:44:49 -0500 From: Benjamin LaHaise To: Linus Torvalds Cc: Dave Chinner , linux-aio@kvack.org, linux-fsdevel , Linux Kernel Mailing List , Linux API , linux-mm , Alexander Viro , Andrew Morton Subject: Re: [PATCH 07/13] aio: enabled thread based async fsync Message-ID: <20160120204449.GC12249@kvack.org> References: <20160112022548.GD6033@dastard> <20160112033708.GE6033@dastard> <20160115202131.GH6330@kvack.org> <20160120195957.GV6033@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.2i Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The most > > recent high profile project that have been reporting issues with AIO > > on XFS is http://www.scylladb.com/. That project is architected > > around non-blocking AIO for scalability reasons... > > I was more wondering about the new interfaces, making sure that the > feature set actually matches what people want to do.. I suspect this will be an ongoing learning exercise as people start to use the new functionality and find gaps in terms of what is needed. Certainly there is a bunch of stuff we need to add to cover the cases where disk i/o is required. getdents() is one example, but the ABI issues we have with it are somewhat more complicated given the history associated with that interface. > That said, I also agree that it would be interesting to hear what the > performance impact is for existing performance-sensitive users. Could > we make that "aio_may_use_threads()" case be unconditional, making > things simpler? Making it unconditional is a goal, but some work is required before that can be the case. The O_DIRECT issue is one such matter -- it requires some changes to the filesystems to ensure that they adhere to the non-blocking nature of the new interface (ie taking i_mutex is a Bad Thing that users really do not want to be exposed to; if taking it blocks, the code should punt to a helper thread). Additional auditing of some of the read/write implementations is also required, which will likely need some minor changes in things like sysfs and other weird functionality we have. Having the flag reflects that while the functionality is useful, not all of the bugs have been worked out yet. What's the desired approach to merge these changes? Does it make sense to merge what is ready now and prepare the next round of changes for 4.6? Or is it more important to grow things to a more complete state before merging? Regards, -ben > Linus -- "Thought is the essence of where you are now." From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 20 Jan 2016 15:44:49 -0500 From: Benjamin LaHaise To: Linus Torvalds Cc: Dave Chinner , linux-aio@kvack.org, linux-fsdevel , Linux Kernel Mailing List , Linux API , linux-mm , Alexander Viro , Andrew Morton Subject: Re: [PATCH 07/13] aio: enabled thread based async fsync Message-ID: <20160120204449.GC12249@kvack.org> References: <20160112022548.GD6033@dastard> <20160112033708.GE6033@dastard> <20160115202131.GH6330@kvack.org> <20160120195957.GV6033@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-ID: On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The most > > recent high profile project that have been reporting issues with AIO > > on XFS is http://www.scylladb.com/. That project is architected > > around non-blocking AIO for scalability reasons... > > I was more wondering about the new interfaces, making sure that the > feature set actually matches what people want to do.. I suspect this will be an ongoing learning exercise as people start to use the new functionality and find gaps in terms of what is needed. Certainly there is a bunch of stuff we need to add to cover the cases where disk i/o is required. getdents() is one example, but the ABI issues we have with it are somewhat more complicated given the history associated with that interface. > That said, I also agree that it would be interesting to hear what the > performance impact is for existing performance-sensitive users. Could > we make that "aio_may_use_threads()" case be unconditional, making > things simpler? Making it unconditional is a goal, but some work is required before that can be the case. The O_DIRECT issue is one such matter -- it requires some changes to the filesystems to ensure that they adhere to the non-blocking nature of the new interface (ie taking i_mutex is a Bad Thing that users really do not want to be exposed to; if taking it blocks, the code should punt to a helper thread). Additional auditing of some of the read/write implementations is also required, which will likely need some minor changes in things like sysfs and other weird functionality we have. Having the flag reflects that while the functionality is useful, not all of the bugs have been worked out yet. What's the desired approach to merge these changes? Does it make sense to merge what is ready now and prepare the next round of changes for 4.6? Or is it more important to grow things to a more complete state before merging? Regards, -ben > Linus -- "Thought is the essence of where you are now." -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin LaHaise Subject: Re: [PATCH 07/13] aio: enabled thread based async fsync Date: Wed, 20 Jan 2016 15:44:49 -0500 Message-ID: <20160120204449.GC12249@kvack.org> References: <20160112022548.GD6033@dastard> <20160112033708.GE6033@dastard> <20160115202131.GH6330@kvack.org> <20160120195957.GV6033@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: owner-linux-aio@kvack.org To: Linus Torvalds Cc: Dave Chinner , linux-aio@kvack.org, linux-fsdevel , Linux Kernel Mailing List , Linux API , linux-mm , Alexander Viro , Andrew Morton List-Id: linux-api@vger.kernel.org On Wed, Jan 20, 2016 at 12:29:32PM -0800, Linus Torvalds wrote: > On Wed, Jan 20, 2016 at 11:59 AM, Dave Chinner wrote: > >> > >> Are there other users outside of Solace? It would be good to get comments.. > > > > I know of quite a few storage/db products that use AIO. The most > > recent high profile project that have been reporting issues with AIO > > on XFS is http://www.scylladb.com/. That project is architected > > around non-blocking AIO for scalability reasons... > > I was more wondering about the new interfaces, making sure that the > feature set actually matches what people want to do.. I suspect this will be an ongoing learning exercise as people start to use the new functionality and find gaps in terms of what is needed. Certainly there is a bunch of stuff we need to add to cover the cases where disk i/o is required. getdents() is one example, but the ABI issues we have with it are somewhat more complicated given the history associated with that interface. > That said, I also agree that it would be interesting to hear what the > performance impact is for existing performance-sensitive users. Could > we make that "aio_may_use_threads()" case be unconditional, making > things simpler? Making it unconditional is a goal, but some work is required before that can be the case. The O_DIRECT issue is one such matter -- it requires some changes to the filesystems to ensure that they adhere to the non-blocking nature of the new interface (ie taking i_mutex is a Bad Thing that users really do not want to be exposed to; if taking it blocks, the code should punt to a helper thread). Additional auditing of some of the read/write implementations is also required, which will likely need some minor changes in things like sysfs and other weird functionality we have. Having the flag reflects that while the functionality is useful, not all of the bugs have been worked out yet. What's the desired approach to merge these changes? Does it make sense to merge what is ready now and prepare the next round of changes for 4.6? Or is it more important to grow things to a more complete state before merging? Regards, -ben > Linus -- "Thought is the essence of where you are now." -- To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to majordomo@kvack.org. For more info on Linux AIO, see: http://www.kvack.org/aio/ Don't email: aart@kvack.org