From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Thu, 15 Jan 2015 12:43:23 -0500 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: lsf-pc@lists.linux-foundation.org Return-path: Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org I would like to talk about enhancing the user interfaces for doing async buffered disk IO for userspace applications. There's a whole class of distributed web applications (most new applications today) that would benefit from such an API. Most of them today rely on cobbling one together in user space using a threadpool. The current in kernel AIO interfaces that only support DIRECTIO, they were generally designed by and for big database vendors. The consensus is that the current AIO interfaces usually lead to decreased performance for those app. I've been developing a new read syscall that allows non-blocking diskio read (provided that data is in the page cache). It's analogous to what exists today in the network world with recvmsg with MSG_NOWAIT flag. The work has been previously described by LWN here: https://lwn.net/Articles/612483/ Previous attempts (over the last 12+ years) at non-blocking buffered diskio has stalled due to their complexity. I would like to talk about the problem, my solution, and get feedback on the course of action. Over the years I've been building the low level guys of various "web applications". That usually involves async network based applications (epoll based servers) and the biggest pain point for the last 8+ years has been async disk IO. -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Moyer Subject: Re: [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Thu, 15 Jan 2015 13:30:05 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain Cc: lsf-pc@lists.linux-foundation.org, "linux-fsdevel\@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: Milosz Tanski Return-path: In-Reply-To: (Milosz Tanski's message of "Thu, 15 Jan 2015 12:43:23 -0500") Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org Milosz Tanski writes: > I would like to talk about enhancing the user interfaces for doing > async buffered disk IO for userspace applications. There's a whole > class of distributed web applications (most new applications today) > that would benefit from such an API. Most of them today rely on > cobbling one together in user space using a threadpool. > > The current in kernel AIO interfaces that only support DIRECTIO, they > were generally designed by and for big database vendors. The consensus > is that the current AIO interfaces usually lead to decreased > performance for those app. > > I've been developing a new read syscall that allows non-blocking > diskio read (provided that data is in the page cache). It's analogous > to what exists today in the network world with recvmsg with MSG_NOWAIT > flag. The work has been previously described by LWN here: > https://lwn.net/Articles/612483/ > > Previous attempts (over the last 12+ years) at non-blocking buffered > diskio has stalled due to their complexity. I would like to talk about > the problem, my solution, and get feedback on the course of action. This email seems to conflate async I/O and non-blocking I/O. Could you please be more specific about what you're proposing to talk about? Is it just the non-blocking read support? Cheers, Jeff -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kara Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Thu, 15 Jan 2015 23:31:57 +0100 Message-ID: <20150115223157.GB25884@quack.suse.cz> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: Milosz Tanski Return-path: Content-Disposition: inline In-Reply-To: Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Thu 15-01-15 12:43:23, Milosz Tanski wrote: > I would like to talk about enhancing the user interfaces for doing > async buffered disk IO for userspace applications. There's a whole > class of distributed web applications (most new applications today) > that would benefit from such an API. Most of them today rely on > cobbling one together in user space using a threadpool. > > The current in kernel AIO interfaces that only support DIRECTIO, they > were generally designed by and for big database vendors. The consensus > is that the current AIO interfaces usually lead to decreased > performance for those app. > > I've been developing a new read syscall that allows non-blocking > diskio read (provided that data is in the page cache). It's analogous > to what exists today in the network world with recvmsg with MSG_NOWAIT > flag. The work has been previously described by LWN here: > https://lwn.net/Articles/612483/ > > Previous attempts (over the last 12+ years) at non-blocking buffered > diskio has stalled due to their complexity. I would like to talk about > the problem, my solution, and get feedback on the course of action. > > Over the years I've been building the low level guys of various "web > applications". That usually involves async network based applications > (epoll based servers) and the biggest pain point for the last 8+ years > has been async disk IO. Maybe this topic will be sorted out before LSF/MM. I know Andrew had some objections about doc and was suggesting a solution using fincore() (which Christoph refuted as being racy). Also there was a pending question regarding whether the async read in this form will be used by applications. But if it doesn't get sorted out a short session on the pending issues would be probably useful. Honza -- Jan Kara SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Fri, 16 Jan 2015 10:40:23 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: Jeff Moyer Return-path: In-Reply-To: Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Thu, Jan 15, 2015 at 1:30 PM, Jeff Moyer wrote: > Milosz Tanski writes: > >> I would like to talk about enhancing the user interfaces for doing >> async buffered disk IO for userspace applications. There's a whole >> class of distributed web applications (most new applications today) >> that would benefit from such an API. Most of them today rely on >> cobbling one together in user space using a threadpool. >> >> The current in kernel AIO interfaces that only support DIRECTIO, they >> were generally designed by and for big database vendors. The consensus >> is that the current AIO interfaces usually lead to decreased >> performance for those app. >> >> I've been developing a new read syscall that allows non-blocking >> diskio read (provided that data is in the page cache). It's analogous >> to what exists today in the network world with recvmsg with MSG_NOWAIT >> flag. The work has been previously described by LWN here: >> https://lwn.net/Articles/612483/ >> >> Previous attempts (over the last 12+ years) at non-blocking buffered >> diskio has stalled due to their complexity. I would like to talk about >> the problem, my solution, and get feedback on the course of action. > > This email seems to conflate async I/O and non-blocking I/O. Could you > please be more specific about what you're proposing to talk about? Is > it just the non-blocking read support? > > Cheers, > Jeff Jeff, I'm sorry if I wasn't clear, let me restate why we should care and why it matters. The current applications that power the lower levels of the web stacks as generally process streams of network data. Many of them (and the frameworks for building them) are structured as a large async processing loop. Disk IO a big pain point; the way are structured (threadpools for diskio) introduces additional latency. sendfile() is only helpful if you need to do additional processing (say SSL). Non-blocking diskio can help us lower the response latency in those webapps applications in the common cases (cached data, sequential scan). -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Fri, 16 Jan 2015 10:44:12 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: Jan Kara Return-path: In-Reply-To: <20150115223157.GB25884@quack.suse.cz> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara wrote: > On Thu 15-01-15 12:43:23, Milosz Tanski wrote: >> I would like to talk about enhancing the user interfaces for doing >> async buffered disk IO for userspace applications. There's a whole >> class of distributed web applications (most new applications today) >> that would benefit from such an API. Most of them today rely on >> cobbling one together in user space using a threadpool. >> >> The current in kernel AIO interfaces that only support DIRECTIO, they >> were generally designed by and for big database vendors. The consensus >> is that the current AIO interfaces usually lead to decreased >> performance for those app. >> >> I've been developing a new read syscall that allows non-blocking >> diskio read (provided that data is in the page cache). It's analogous >> to what exists today in the network world with recvmsg with MSG_NOWAIT >> flag. The work has been previously described by LWN here: >> https://lwn.net/Articles/612483/ >> >> Previous attempts (over the last 12+ years) at non-blocking buffered >> diskio has stalled due to their complexity. I would like to talk about >> the problem, my solution, and get feedback on the course of action. >> >> Over the years I've been building the low level guys of various "web >> applications". That usually involves async network based applications >> (epoll based servers) and the biggest pain point for the last 8+ years >> has been async disk IO. > Maybe this topic will be sorted out before LSF/MM. I know Andrew had some > objections about doc and was suggesting a solution using fincore() (which > Christoph refuted as being racy). Also there was a pending question > regarding whether the async read in this form will be used by applications. > But if it doesn't get sorted out a short session on the pending issues > would be probably useful. > > Honza > -- > Jan Kara > SUSE Labs, CR I've spent the better part of yesterday wrapping up the first cut of samba support to FIO so we can test a modified samba file server with these changes in a few scenarios. Right now it's only sync but I hope to have async in the future. I hope that by the time the summit rolls around I'll have data to share from samba and maybe some other common apps (node.js / twisted). -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Fri, 16 Jan 2015 08:55:06 -0800 Message-ID: <20150116165506.GA10856@samba2> References: <20150115223157.GB25884@quack.suse.cz> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jan Kara , lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig To: Milosz Tanski Return-path: Received: from fn.samba.org ([216.83.154.106]:49916 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753014AbbAPQzJ (ORCPT ); Fri, 16 Jan 2015 11:55:09 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote: > On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara wrote: > > On Thu 15-01-15 12:43:23, Milosz Tanski wrote: > >> I would like to talk about enhancing the user interfaces for doing > >> async buffered disk IO for userspace applications. There's a whole > >> class of distributed web applications (most new applications today) > >> that would benefit from such an API. Most of them today rely on > >> cobbling one together in user space using a threadpool. > >> > >> The current in kernel AIO interfaces that only support DIRECTIO, they > >> were generally designed by and for big database vendors. The consensus > >> is that the current AIO interfaces usually lead to decreased > >> performance for those app. > >> > >> I've been developing a new read syscall that allows non-blocking > >> diskio read (provided that data is in the page cache). It's analogous > >> to what exists today in the network world with recvmsg with MSG_NOWAIT > >> flag. The work has been previously described by LWN here: > >> https://lwn.net/Articles/612483/ > >> > >> Previous attempts (over the last 12+ years) at non-blocking buffered > >> diskio has stalled due to their complexity. I would like to talk about > >> the problem, my solution, and get feedback on the course of action. > >> > >> Over the years I've been building the low level guys of various "web > >> applications". That usually involves async network based applications > >> (epoll based servers) and the biggest pain point for the last 8+ years > >> has been async disk IO. > > Maybe this topic will be sorted out before LSF/MM. I know Andrew had some > > objections about doc and was suggesting a solution using fincore() (which > > Christoph refuted as being racy). Also there was a pending question > > regarding whether the async read in this form will be used by applications. > > But if it doesn't get sorted out a short session on the pending issues > > would be probably useful. > > > > Honza > > -- > > Jan Kara > > SUSE Labs, CR > > I've spent the better part of yesterday wrapping up the first cut of > samba support to FIO so we can test a modified samba file server with > these changes in a few scenarios. Right now it's only sync but I hope > to have async in the future. I hope that by the time the summit rolls > around I'll have data to share from samba and maybe some other common > apps (node.js / twisted). Don't forget to share the code changes :-). We @ Samba would love to see them to keep track ! From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Sun, 18 Jan 2015 22:49:36 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: Jan Kara , lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig , Volker Lendecke , Jens Axboe To: Jeremy Allison Return-path: In-Reply-To: <20150116165506.GA10856@samba2> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Fri, Jan 16, 2015 at 11:55 AM, Jeremy Allison wrote: > On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote: >> On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara wrote: >> > On Thu 15-01-15 12:43:23, Milosz Tanski wrote: >> >> I would like to talk about enhancing the user interfaces for doing >> >> async buffered disk IO for userspace applications. There's a whole >> >> class of distributed web applications (most new applications today) >> >> that would benefit from such an API. Most of them today rely on >> >> cobbling one together in user space using a threadpool. >> >> >> >> The current in kernel AIO interfaces that only support DIRECTIO, they >> >> were generally designed by and for big database vendors. The consensus >> >> is that the current AIO interfaces usually lead to decreased >> >> performance for those app. >> >> >> >> I've been developing a new read syscall that allows non-blocking >> >> diskio read (provided that data is in the page cache). It's analogous >> >> to what exists today in the network world with recvmsg with MSG_NOWAIT >> >> flag. The work has been previously described by LWN here: >> >> https://lwn.net/Articles/612483/ >> >> >> >> Previous attempts (over the last 12+ years) at non-blocking buffered >> >> diskio has stalled due to their complexity. I would like to talk about >> >> the problem, my solution, and get feedback on the course of action. >> >> >> >> Over the years I've been building the low level guys of various "web >> >> applications". That usually involves async network based applications >> >> (epoll based servers) and the biggest pain point for the last 8+ years >> >> has been async disk IO. >> > Maybe this topic will be sorted out before LSF/MM. I know Andrew had some >> > objections about doc and was suggesting a solution using fincore() (which >> > Christoph refuted as being racy). Also there was a pending question >> > regarding whether the async read in this form will be used by applications. >> > But if it doesn't get sorted out a short session on the pending issues >> > would be probably useful. >> > >> > Honza >> > -- >> > Jan Kara >> > SUSE Labs, CR >> >> I've spent the better part of yesterday wrapping up the first cut of >> samba support to FIO so we can test a modified samba file server with >> these changes in a few scenarios. Right now it's only sync but I hope >> to have async in the future. I hope that by the time the summit rolls >> around I'll have data to share from samba and maybe some other common >> apps (node.js / twisted). > > Don't forget to share the code changes :-). We @ Samba would > love to see them to keep track ! I have the first version of the FIO cifs support via samba in my fork of FIO here: https://github.com/mtanski/fio/tree/samba Right now it only supports sync mode of FIO (eg. can't submit multiple outstanding requests) but I'm looking into how to make it work with smb2 read/write calls with the async flag. Additionally, I'm sure I'm doing some things not quite right in terms of smbcli usage as it was a decent amount of trial and error to get it to connect (esp. the setup before smbcli_full_connection). Finally, it looks like the more complex api I'm using (as opposed to smbclient, because I want the async calls) doesn't quite fully export all calls I need via headers / public dyn libs so it's a bit of a hack to get it to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 But it works for my randread tests with zipf and the great part is that it should provide a flexible way to test samba with many fake clients and access patterns. So... progress. -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Sun, 18 Jan 2015 23:12:18 -0800 Message-ID: <20150119071218.GA9747@jeremy-HP> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jeremy Allison , Jan Kara , lsf-pc@lists.linux-foundation.org, "linux-fsdevel@vger.kernel.org" , linux-mm@kvack.org, Christoph Hellwig , Volker Lendecke , Jens Axboe To: Milosz Tanski Return-path: Received: from fn.samba.org ([216.83.154.106]:33122 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750968AbbASHMQ (ORCPT ); Mon, 19 Jan 2015 02:12:16 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote: > > I have the first version of the FIO cifs support via samba in my fork > of FIO here: https://github.com/mtanski/fio/tree/samba > > Right now it only supports sync mode of FIO (eg. can't submit multiple > outstanding requests) but I'm looking into how to make it work with > smb2 read/write calls with the async flag. > > Additionally, I'm sure I'm doing some things not quite right in terms > of smbcli usage as it was a decent amount of trial and error to get it > to connect (esp. the setup before smbcli_full_connection). Finally, it > looks like the more complex api I'm using (as opposed to smbclient, > because I want the async calls) doesn't quite fully export all calls I > need via headers / public dyn libs so it's a bit of a hack to get it > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 > > But it works for my randread tests with zipf and the great part is > that it should provide a flexible way to test samba with many fake > clients and access patterns. So... progress. One problem here. Looks like fio is under GPLv2-only, is that correct ? If so there's no way to combine the two codebases, as Samba is under GPLv3-or-later with parts under LGPLv3-or-later. fio needs to be GPLv2-or-later in order to be able to use with libsmbclient. Cheers, Jeremy. From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Bottomley Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Sun, 18 Jan 2015 23:34:09 -0800 Message-ID: <1421652849.2080.20.camel@HansenPartnership.com> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Milosz Tanski , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jeremy Allison Return-path: In-Reply-To: <20150119071218.GA9747@jeremy-HP> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote: > On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote: > > > > I have the first version of the FIO cifs support via samba in my fork > > of FIO here: https://github.com/mtanski/fio/tree/samba > > > > Right now it only supports sync mode of FIO (eg. can't submit multiple > > outstanding requests) but I'm looking into how to make it work with > > smb2 read/write calls with the async flag. > > > > Additionally, I'm sure I'm doing some things not quite right in terms > > of smbcli usage as it was a decent amount of trial and error to get it > > to connect (esp. the setup before smbcli_full_connection). Finally, it > > looks like the more complex api I'm using (as opposed to smbclient, > > because I want the async calls) doesn't quite fully export all calls I > > need via headers / public dyn libs so it's a bit of a hack to get it > > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 > > > > But it works for my randread tests with zipf and the great part is > > that it should provide a flexible way to test samba with many fake > > clients and access patterns. So... progress. > > One problem here. Looks like fio is under GPLv2-only, > is that correct ? Seems so from the LICENSE file. > If so there's no way to combine the two codebases, > as Samba is under GPLv3-or-later with parts under LGPLv3-or-later. > > fio needs to be GPLv2-or-later in order to be > able to use with libsmbclient. That's one of these pointless licensing complexities that annoy distributions so much ... they're both open source, so there's no real problem except the licence incompatibility. The usual way out of it is just to dual licence the incompatible component. James -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:18:49 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: Jeremy Allison , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: James Bottomley Return-path: In-Reply-To: <1421652849.2080.20.camel@HansenPartnership.com> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley wrote: > On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote: >> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote: >> > >> > I have the first version of the FIO cifs support via samba in my fork >> > of FIO here: https://github.com/mtanski/fio/tree/samba >> > >> > Right now it only supports sync mode of FIO (eg. can't submit multiple >> > outstanding requests) but I'm looking into how to make it work with >> > smb2 read/write calls with the async flag. >> > >> > Additionally, I'm sure I'm doing some things not quite right in terms >> > of smbcli usage as it was a decent amount of trial and error to get it >> > to connect (esp. the setup before smbcli_full_connection). Finally, it >> > looks like the more complex api I'm using (as opposed to smbclient, >> > because I want the async calls) doesn't quite fully export all calls I >> > need via headers / public dyn libs so it's a bit of a hack to get it >> > to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 >> > >> > But it works for my randread tests with zipf and the great part is >> > that it should provide a flexible way to test samba with many fake >> > clients and access patterns. So... progress. >> >> One problem here. Looks like fio is under GPLv2-only, >> is that correct ? > > Seems so from the LICENSE file. > >> If so there's no way to combine the two codebases, >> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later. >> >> fio needs to be GPLv2-or-later in order to be >> able to use with libsmbclient. > > That's one of these pointless licensing complexities that annoy > distributions so much ... they're both open source, so there's no real > problem except the licence incompatibility. The usual way out of it is > just to dual licence the incompatible component. > > James > > Sadly, in this case there's nothing I can do about the license; both projects have a right to determine their own licensing. Hopefully, the parties can come to some kind of agreement since it would be beneficial to use fio to test samba. This works well enough for me to test test preadv2 using samba and get numbers. So I'll use this to do some preadv2 testing using samba for different workloads. -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 08:31:27 -0700 Message-ID: <54BD234F.3060203@kernel.dk> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Jeremy Allison , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Milosz Tanski , James Bottomley Return-path: Received: from mail-pd0-f171.google.com ([209.85.192.171]:49282 "EHLO mail-pd0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751467AbbASPbk (ORCPT ); Mon, 19 Jan 2015 10:31:40 -0500 Received: by mail-pd0-f171.google.com with SMTP id fp1so15779982pdb.2 for ; Mon, 19 Jan 2015 07:31:40 -0800 (PST) In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On 01/19/2015 07:18 AM, Milosz Tanski wrote: > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley > wrote: >> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote: >>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote: >>>> >>>> I have the first version of the FIO cifs support via samba in my fork >>>> of FIO here: https://github.com/mtanski/fio/tree/samba >>>> >>>> Right now it only supports sync mode of FIO (eg. can't submit multiple >>>> outstanding requests) but I'm looking into how to make it work with >>>> smb2 read/write calls with the async flag. >>>> >>>> Additionally, I'm sure I'm doing some things not quite right in terms >>>> of smbcli usage as it was a decent amount of trial and error to get it >>>> to connect (esp. the setup before smbcli_full_connection). Finally, it >>>> looks like the more complex api I'm using (as opposed to smbclient, >>>> because I want the async calls) doesn't quite fully export all calls I >>>> need via headers / public dyn libs so it's a bit of a hack to get it >>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 >>>> >>>> But it works for my randread tests with zipf and the great part is >>>> that it should provide a flexible way to test samba with many fake >>>> clients and access patterns. So... progress. >>> >>> One problem here. Looks like fio is under GPLv2-only, >>> is that correct ? >> >> Seems so from the LICENSE file. >> >>> If so there's no way to combine the two codebases, >>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later. >>> >>> fio needs to be GPLv2-or-later in order to be >>> able to use with libsmbclient. >> >> That's one of these pointless licensing complexities that annoy >> distributions so much ... they're both open source, so there's no real >> problem except the licence incompatibility. The usual way out of it is >> just to dual licence the incompatible component. >> >> James >> >> > > Sadly, in this case there's nothing I can do about the license; both > projects have a right to determine their own licensing. Hopefully, the > parties can come to some kind of agreement since it would be > beneficial to use fio to test samba. > > This works well enough for me to test test preadv2 using samba and get > numbers. So I'll use this to do some preadv2 testing using samba for > different workloads. I didn't look at your code yet, but I'm assuming it's a self contained IO engine. So we should be able to make that work, by only linking the engine itself against libsmbclient. But sheesh, what a pain in the butt, why can't we just all be friends. So don't worry about licensing for now, just work on improving the engine and we'll sort the non-technical details out. -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Bottomley Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 07:49:41 -0800 Message-ID: <1421682581.2080.22.camel@HansenPartnership.com> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Milosz Tanski , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org, Jeremy Allison To: Jens Axboe Return-path: Received: from bedivere.hansenpartnership.com ([66.63.167.143]:51781 "EHLO bedivere.hansenpartnership.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751993AbbASPtn (ORCPT ); Mon, 19 Jan 2015 10:49:43 -0500 In-Reply-To: <54BD234F.3060203@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, 2015-01-19 at 08:31 -0700, Jens Axboe wrote: > On 01/19/2015 07:18 AM, Milosz Tanski wrote: > > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley > > wrote: > >> On Sun, 2015-01-18 at 23:12 -0800, Jeremy Allison wrote: > >>> On Sun, Jan 18, 2015 at 10:49:36PM -0500, Milosz Tanski wrote: > >>>> > >>>> I have the first version of the FIO cifs support via samba in my fork > >>>> of FIO here: https://github.com/mtanski/fio/tree/samba > >>>> > >>>> Right now it only supports sync mode of FIO (eg. can't submit multiple > >>>> outstanding requests) but I'm looking into how to make it work with > >>>> smb2 read/write calls with the async flag. > >>>> > >>>> Additionally, I'm sure I'm doing some things not quite right in terms > >>>> of smbcli usage as it was a decent amount of trial and error to get it > >>>> to connect (esp. the setup before smbcli_full_connection). Finally, it > >>>> looks like the more complex api I'm using (as opposed to smbclient, > >>>> because I want the async calls) doesn't quite fully export all calls I > >>>> need via headers / public dyn libs so it's a bit of a hack to get it > >>>> to build: https://github.com/mtanski/fio/commit/7fd35359259b409ed023b924cb2758e9efb9950c#diff-1 > >>>> > >>>> But it works for my randread tests with zipf and the great part is > >>>> that it should provide a flexible way to test samba with many fake > >>>> clients and access patterns. So... progress. > >>> > >>> One problem here. Looks like fio is under GPLv2-only, > >>> is that correct ? > >> > >> Seems so from the LICENSE file. > >> > >>> If so there's no way to combine the two codebases, > >>> as Samba is under GPLv3-or-later with parts under LGPLv3-or-later. > >>> > >>> fio needs to be GPLv2-or-later in order to be > >>> able to use with libsmbclient. > >> > >> That's one of these pointless licensing complexities that annoy > >> distributions so much ... they're both open source, so there's no real > >> problem except the licence incompatibility. The usual way out of it is > >> just to dual licence the incompatible component. > >> > >> James > >> > >> > > > > Sadly, in this case there's nothing I can do about the license; both > > projects have a right to determine their own licensing. Hopefully, the > > parties can come to some kind of agreement since it would be > > beneficial to use fio to test samba. > > > > This works well enough for me to test test preadv2 using samba and get > > numbers. So I'll use this to do some preadv2 testing using samba for > > different workloads. > > I didn't look at your code yet, but I'm assuming it's a self contained > IO engine. So we should be able to make that work, by only linking the > engine itself against libsmbclient. But sheesh, what a pain in the butt, > why can't we just all be friends. > > So don't worry about licensing for now, just work on improving the > engine and we'll sort the non-technical details out. For fio, it likely doesn't matter. Most people download the repository and compile it themselves when building the tool. In that case, there's no licence violation anyway (all GPL issues, including technical licence incompatibility, manifest on distribution not on use). It is a problem for the distributors, but they're well used to these type of self inflicted wounds. James From mboxrd@z Thu Jan 1 00:00:00 1970 From: Volker Lendecke Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 17:10:03 +0100 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> Reply-To: Volker.Lendecke@SerNet.DE Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Milosz Tanski , James Bottomley , Jeremy Allison , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jens Axboe Return-path: Content-Disposition: inline In-Reply-To: <54BD234F.3060203@kernel.dk> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote: > I didn't look at your code yet, but I'm assuming it's a self > contained IO engine. So we should be able to make that work, by only > linking the engine itself against libsmbclient. But sheesh, what a > pain in the butt, why can't we just all be friends. The published libsmbclient API misses the async features that are needed here. Milosz needs to go lower-level. Volker --=20 SerNet GmbH, Bahnhofsallee 1b, 37081 G=F6ttingen phone: +49-551-370000-0, fax: +49-551-370000-9 AG G=F6ttingen, HRB 2816, GF: Dr. Johannes Loxen http://www.sernet.de, mailto:kontakt@sernet.de -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 11:20:50 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: Jens Axboe , James Bottomley , Jeremy Allison , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Volker Lendecke Return-path: Received: from mail-lb0-f180.google.com ([209.85.217.180]:54334 "EHLO mail-lb0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751143AbbASQUx (ORCPT ); Mon, 19 Jan 2015 11:20:53 -0500 Received: by mail-lb0-f180.google.com with SMTP id b6so3527674lbj.11 for ; Mon, 19 Jan 2015 08:20:50 -0800 (PST) In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Jan 19, 2015 at 11:10 AM, Volker Lendecke wrote: > On Mon, Jan 19, 2015 at 08:31:27AM -0700, Jens Axboe wrote: >> I didn't look at your code yet, but I'm assuming it's a self >> contained IO engine. So we should be able to make that work, by only >> linking the engine itself against libsmbclient. But sheesh, what a >> pain in the butt, why can't we just all be friends. > > The published libsmbclient API misses the async features > that are needed here. Milosz needs to go lower-level. > > Volker Volker, the sync code path works; in fact I pushed some minor corrections to my branch this morning. And for now using FIO I can generate multiple clients (threads / processes). I started working on the async features (SMB2 async read/write) for client library the samba repo. There's a patch there for the first step of it it there; see the other email I sent to you and Jeremy. I was going to make sure it licensed under whatever it needs to get into the samba repo... and since this is done on my own time I personally don't care what license it's under provided it's not a PITA. -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 08:48:57 -0800 Message-ID: <20150119164857.GC12308@jeremy-HP> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <1421682581.2080.22.camel@HansenPartnership.com> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jens Axboe , Milosz Tanski , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org, Jeremy Allison To: James Bottomley Return-path: Received: from fn.samba.org ([216.83.154.106]:50311 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751535AbbASQsx (ORCPT ); Mon, 19 Jan 2015 11:48:53 -0500 Content-Disposition: inline In-Reply-To: <1421682581.2080.22.camel@HansenPartnership.com> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote: > > For fio, it likely doesn't matter. Most people download the repository > and compile it themselves when building the tool. In that case, there's > no licence violation anyway (all GPL issues, including technical licence > incompatibility, manifest on distribution not on use). It is a problem > for the distributors, but they're well used to these type of self > inflicted wounds. That's true, but it is setting a bear-trap for distributors. Might be better to keep the code repositories separate so at lease people have a *chance* of noticing there's a problem here. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 08:50:30 -0800 Message-ID: <20150119165030.GD12308@jeremy-HP> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk>

Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Volker Lendecke , Jens Axboe , James Bottomley , Jeremy Allison , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Milosz Tanski Return-path: Received: from fn.samba.org ([216.83.154.106]:50414 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751520AbbASQu0 (ORCPT ); Mon, 19 Jan 2015 11:50:26 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Jan 19, 2015 at 11:20:50AM -0500, Milosz Tanski wrote: > > Volker, the sync code path works; in fact I pushed some minor > corrections to my branch this morning. And for now using FIO I can > generate multiple clients (threads / processes). > > I started working on the async features (SMB2 async read/write) for > client library the samba repo. There's a patch there for the first > step of it it there; see the other email I sent to you and Jeremy. I > was going to make sure it licensed under whatever it needs to get into > the samba repo... and since this is done on my own time I personally > don't care what license it's under provided it's not a PITA. Anything going into Samba would need to be permissively licensed (MIT/BSD) or GPLv3+ or LGPLv3+. We'd prefer the latter, but we're happy with either. The one thing it *can't* be though, is GPLv2-only. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:04:29 -0800 Message-ID: <20150119170429.GA13160@jeremy-HP> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: James Bottomley , Jeremy Allison , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Milosz Tanski Return-path: Received: from fn.samba.org ([216.83.154.106]:50771 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751374AbbASREZ (ORCPT ); Mon, 19 Jan 2015 12:04:25 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley wrote: > > That's one of these pointless licensing complexities that annoy > distributions so much ... they're both open source, so there's no real > problem except the licence incompatibility. The usual way out of it is > just to dual licence the incompatible component. Just one point here - we're not able to dual license Samba to go back to GPLv2 anything. There are too many contributors to this who have contributed under v3-or-later licensing in order for this to be possible for us. I'm hoping adding the 'or-later' clause to fio might be easier. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Richard Sharpe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:18:14 -0800 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <20150119170429.GA13160@jeremy-HP> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Milosz Tanski , James Bottomley , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jeremy Allison Return-path: In-Reply-To: <20150119170429.GA13160@jeremy-HP> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 19, 2015 at 9:04 AM, Jeremy Allison wrote: > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley > wrote: >> >> That's one of these pointless licensing complexities that annoy >> distributions so much ... they're both open source, so there's no real >> problem except the licence incompatibility. The usual way out of it is >> just to dual licence the incompatible component. > > Just one point here - we're not able to dual license > Samba to go back to GPLv2 anything. There are too many > contributors to this who have contributed under v3-or-later > licensing in order for this to be possible for us. > > I'm hoping adding the 'or-later' clause to fio might > be easier. As someone who has worked for companies that distribute Samba for quite a while I cannot see us distributing fio. Rather, we would use it as a performance testing tool. That being the case, the license differences are not a problem. Am I missing something here? --=20 Regards, Richard Sharpe (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D=9C= =E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:20:15 -0800 Message-ID: <20150119172015.GA13428@jeremy-HP> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <20150119170429.GA13160@jeremy-HP> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jeremy Allison , Milosz Tanski , James Bottomley , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Richard Sharpe Return-path: Received: from fn.samba.org ([216.83.154.106]:51285 "EHLO mail.samba.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751557AbbASRUL (ORCPT ); Mon, 19 Jan 2015 12:20:11 -0500 Content-Disposition: inline In-Reply-To: Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Mon, Jan 19, 2015 at 09:18:14AM -0800, Richard Sharpe wrote: > On Mon, Jan 19, 2015 at 9:04 AM, Jeremy Allison wrote: > > On Mon, Jan 19, 2015 at 2:34 AM, James Bottomley > > wrote: > >> > >> That's one of these pointless licensing complexities that annoy > >> distributions so much ... they're both open source, so there's no real > >> problem except the licence incompatibility. The usual way out of it is > >> just to dual licence the incompatible component. > > > > Just one point here - we're not able to dual license > > Samba to go back to GPLv2 anything. There are too many > > contributors to this who have contributed under v3-or-later > > licensing in order for this to be possible for us. > > > > I'm hoping adding the 'or-later' clause to fio might > > be easier. > > As someone who has worked for companies that distribute Samba for > quite a while I cannot see us distributing fio. Rather, we would use > it as a performance testing tool. > > That being the case, the license differences are not a problem. > > Am I missing something here? No, it's only a problem for distributors, so it's much less of a problem than it might be. But it's still a problem I'd rather not have to think about :-). From mboxrd@z Thu Jan 1 00:00:00 1970 From: James Bottomley Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:26:54 -0800 Message-ID: <1421688414.2080.38.camel@HansenPartnership.com> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <1421682581.2080.22.camel@HansenPartnership.com> <20150119164857.GC12308@jeremy-HP> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, Milosz Tanski , "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jeremy Allison Return-path: In-Reply-To: <20150119164857.GC12308@jeremy-HP> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, 2015-01-19 at 08:48 -0800, Jeremy Allison wrote: > On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote: > > > > For fio, it likely doesn't matter. Most people download the repository > > and compile it themselves when building the tool. In that case, there's > > no licence violation anyway (all GPL issues, including technical licence > > incompatibility, manifest on distribution not on use). It is a problem > > for the distributors, but they're well used to these type of self > > inflicted wounds. > > That's true, but it is setting a bear-trap for distributors. It's hardly a bear trap ... this type of annoyance is what they're used to. Some even just ignore it on the grounds of no harm no foul. The first thing they'll ask when they notice is for the protagonists to dual licence. > Might be better to keep the code repositories separate so at > lease people have a *chance* of noticing there's a problem > here. Actually, it might be better to *resolve* the problem before people notice ... if the combination is considered useful, of course. James -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeremy Allison Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:32:57 -0800 Message-ID: <20150119173257.GA14079@jeremy-HP> References: <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <1421682581.2080.22.camel@HansenPartnership.com> <20150119164857.GC12308@jeremy-HP> <1421688414.2080.38.camel@HansenPartnership.com> Reply-To: Jeremy Allison Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jeremy Allison , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, Milosz Tanski , "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: James Bottomley Return-path: Content-Disposition: inline In-Reply-To: <1421688414.2080.38.camel@HansenPartnership.com> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 19, 2015 at 09:26:54AM -0800, James Bottomley wrote: > > Actually, it might be better to *resolve* the problem before people > notice ... if the combination is considered useful, of course. Oh sure - no arguments from me there ! -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Richard Sharpe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 09:36:00 -0800 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <1421682581.2080.22.camel@HansenPartnership.com> <20150119164857.GC12308@jeremy-HP> <1421688414.2080.38.camel@HansenPartnership.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Jeremy Allison , Jens Axboe , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, Milosz Tanski , "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: James Bottomley Return-path: In-Reply-To: <1421688414.2080.38.camel@HansenPartnership.com> Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org On Mon, Jan 19, 2015 at 9:26 AM, James Bottomley wrote: > On Mon, 2015-01-19 at 08:48 -0800, Jeremy Allison wrote: >> On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote: >> > >> > For fio, it likely doesn't matter. Most people download the repositor= y >> > and compile it themselves when building the tool. In that case, there'= s >> > no licence violation anyway (all GPL issues, including technical licen= ce >> > incompatibility, manifest on distribution not on use). It is a proble= m >> > for the distributors, but they're well used to these type of self >> > inflicted wounds. >> >> That's true, but it is setting a bear-trap for distributors. > > It's hardly a bear trap ... this type of annoyance is what they're used > to. Some even just ignore it on the grounds of no harm no foul. The > first thing they'll ask when they notice is for the protagonists to dual > licence. > >> Might be better to keep the code repositories separate so at >> lease people have a *chance* of noticing there's a problem >> here. > > Actually, it might be better to *resolve* the problem before people > notice ... if the combination is considered useful, of course. Actually, I retract my earlier comment. The combination could be very useful to some companies that ship Samba. I can see it being used by field support to test whether there are performance problems either from Windows (under cygwin) or from Linux. That being the case, it would be useful to resolve the license issue before it becomes an issue. --=20 Regards, Richard Sharpe (=E4=BD=95=E4=BB=A5=E8=A7=A3=E6=86=82=EF=BC=9F=E5=94=AF=E6=9C=89=E6=9D=9C= =E5=BA=B7=E3=80=82--=E6=9B=B9=E6=93=8D) -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Mon, 19 Jan 2015 13:00:39 -0700 Message-ID: <54BD6267.1090106@kernel.dk> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <1421682581.2080.22.camel@HansenPartnership.com> <20150119164857.GC12308@jeremy-HP> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Milosz Tanski , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jeremy Allison , James Bottomley Return-path: Received: from mail-pa0-f50.google.com ([209.85.220.50]:47252 "EHLO mail-pa0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751849AbbASUAm (ORCPT ); Mon, 19 Jan 2015 15:00:42 -0500 Received: by mail-pa0-f50.google.com with SMTP id bj1so40682358pad.9 for ; Mon, 19 Jan 2015 12:00:42 -0800 (PST) In-Reply-To: <20150119164857.GC12308@jeremy-HP> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On 01/19/2015 09:48 AM, Jeremy Allison wrote: > On Mon, Jan 19, 2015 at 07:49:41AM -0800, James Bottomley wrote: >> >> For fio, it likely doesn't matter. Most people download the repository >> and compile it themselves when building the tool. In that case, there's >> no licence violation anyway (all GPL issues, including technical licence >> incompatibility, manifest on distribution not on use). It is a problem >> for the distributors, but they're well used to these type of self >> inflicted wounds. > > That's true, but it is setting a bear-trap for distributors. But not unique. Most distros are behind on fio anyway, so most people do end up compiling on their own. > Might be better to keep the code repositories separate so at > lease people have a *chance* of noticing there's a problem > here. But that's a pain for users, I'd much rather include it and let the distro sort it. They can just add --disable-smb or something to their configure scripts. -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Tue, 20 Jan 2015 12:33:22 -0700 Message-ID: <54BEAD82.3070501@kernel.dk> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Jeremy Allison , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Milosz Tanski , James Bottomley Return-path: Received: from mail-pa0-f43.google.com ([209.85.220.43]:63894 "EHLO mail-pa0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751557AbbATTdZ (ORCPT ); Tue, 20 Jan 2015 14:33:25 -0500 Received: by mail-pa0-f43.google.com with SMTP id eu11so10328061pac.2 for ; Tue, 20 Jan 2015 11:33:24 -0800 (PST) In-Reply-To: <54BD234F.3060203@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On 01/19/2015 08:31 AM, Jens Axboe wrote: > I didn't look at your code yet, but I'm assuming it's a self contained > IO engine. So we should be able to make that work, by only linking the > engine itself against libsmbclient. But sheesh, what a pain in the butt, > why can't we just all be friends. I pulled it in for testing, and came up with this patch [1]. If you don't do anything, it'll build cifs into fio as before. If you add --cifs-external to the configure arguments, it'll build cifs.so as an externally loadable module. You'd then use: ioengine=/path/to/cifs.so to use that module. I did not add an install target, I'll leave that to distros... Let me know how that works for you. And let me know how far along you are with the cifs engine, I'd like to pull it in. http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs [1] http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729 -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Tue, 20 Jan 2015 18:07:51 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <54BEAD82.3070501@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: James Bottomley , Jeremy Allison , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jens Axboe Return-path: Received: from mail-lb0-f170.google.com ([209.85.217.170]:43916 "EHLO mail-lb0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751614AbbATXHx (ORCPT ); Tue, 20 Jan 2015 18:07:53 -0500 Received: by mail-lb0-f170.google.com with SMTP id 10so36180971lbg.1 for ; Tue, 20 Jan 2015 15:07:51 -0800 (PST) In-Reply-To: <54BEAD82.3070501@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Great, I'll pull into my branch. I'm already using FIO with the cifs engine to test and help me debug preadv2 changes to smbd and it works without issues for me today. I'm going to work on the async cifs engine but that will take longer because I need to expose build the async SMB2 support into libsmbclient. I'm going to leave that work till after I get further with preadv2 in samba (hopefully this week). I think the biggest issue with the changes is the configure part of my changes (as seen here: http://git.kernel.dk/?p=fio.git;a=blobdiff;f=configure;h=d4502095250cdf5187b24276c327b727d3d87288;hp=33d1327ebbba5b70a001e422bb5ad9b24d7c7b49;hb=7fd35359259b409ed023b924cb2758e9efb9950c;hpb=5fb4b36674b194ae6c6756314dc0c665fcaea06d ). The way samba packages the client libraries beyond just smbclient-raw requiring me to pull in other libraries and then mess with rpath to guess the distro location is far for ideal. I haven't reported it yet mostly because I was interested it making progress and making it work. Ideally samba folks would fix the pkgconfig file for smbclient-raw to set right LDPATH (including all the depending libraries and rpath) so that stuff is not needed. Additionally, there's a few things not exported in the header files (but used) like: http://git.kernel.dk/?p=fio.git;a=blob;f=engines/cifs.c;h=67c23fac0c70cfc75932c758f44dd377fc3f2608;hb=7fd35359259b409ed023b924cb2758e9efb9950c#l16 . It looks like lpcfg_resolve_context() is the only way to create a struct resolve_context which is used in the cliraw hreads, but lpcfg_resolve_context() is not exported via the header files. Some of this might be not using the library correctly... there really wasn't any documentation so I just guessed by looking the torture code in samba and the smbclient and to see what order to punch the lpcfg_stuff to make smbcli_full_connection() not fail. On Tue, Jan 20, 2015 at 2:33 PM, Jens Axboe wrote: > On 01/19/2015 08:31 AM, Jens Axboe wrote: >> >> I didn't look at your code yet, but I'm assuming it's a self contained >> IO engine. So we should be able to make that work, by only linking the >> engine itself against libsmbclient. But sheesh, what a pain in the butt, >> why can't we just all be friends. > > > I pulled it in for testing, and came up with this patch [1]. If you don't do > anything, it'll build cifs into fio as before. If you add --cifs-external to > the configure arguments, it'll build cifs.so as an externally loadable > module. You'd then use: > > ioengine=/path/to/cifs.so > > to use that module. I did not add an install target, I'll leave that to > distros... > > Let me know how that works for you. And let me know how far along you are > with the cifs engine, I'd like to pull it in. > > http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs > > [1] > http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729 > > -- > Jens Axboe > -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com From mboxrd@z Thu Jan 1 00:00:00 1970 From: Milosz Tanski Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Tue, 20 Jan 2015 18:22:08 -0500 Message-ID: References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <54BEAD82.3070501@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: James Bottomley , Jeremy Allison , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Jens Axboe Return-path: In-Reply-To: Sender: owner-linux-mm@kvack.org List-Id: linux-fsdevel.vger.kernel.org Side note Jens. Can you add a configure flag to disable use of SHM (like for ESX)? It took me a while to figure out the proper define to manually stick in the configure. The motivation for this is using rr (mozila's replay debugger) to debug fio. rr doesn't support SHM. http://rr-project.org/ gdb's reversible debugging is too painfully slow. Thanks, - Milosz On Tue, Jan 20, 2015 at 6:07 PM, Milosz Tanski wrote: > Great, I'll pull into my branch. I'm already using FIO with the cifs > engine to test and help me debug preadv2 changes to smbd and it works > without issues for me today. I'm going to work on the async cifs > engine but that will take longer because I need to expose build the > async SMB2 support into libsmbclient. I'm going to leave that work > till after I get further with preadv2 in samba (hopefully this week). > > I think the biggest issue with the changes is the configure part of my > changes (as seen here: > http://git.kernel.dk/?p=fio.git;a=blobdiff;f=configure;h=d4502095250cdf5187b24276c327b727d3d87288;hp=33d1327ebbba5b70a001e422bb5ad9b24d7c7b49;hb=7fd35359259b409ed023b924cb2758e9efb9950c;hpb=5fb4b36674b194ae6c6756314dc0c665fcaea06d > ). > > The way samba packages the client libraries beyond just smbclient-raw > requiring me to pull in other libraries and then mess with rpath to > guess the distro location is far for ideal. I haven't reported it yet > mostly because I was interested it making progress and making it work. > Ideally samba folks would fix the pkgconfig file for smbclient-raw to > set right LDPATH (including all the depending libraries and rpath) so > that stuff is not needed. > > Additionally, there's a few things not exported in the header files > (but used) like: > http://git.kernel.dk/?p=fio.git;a=blob;f=engines/cifs.c;h=67c23fac0c70cfc75932c758f44dd377fc3f2608;hb=7fd35359259b409ed023b924cb2758e9efb9950c#l16 > . It looks like lpcfg_resolve_context() is the only way to create a > struct resolve_context which is used in the cliraw hreads, but > lpcfg_resolve_context() is not exported via the header files. > > Some of this might be not using the library correctly... there really > wasn't any documentation so I just guessed by looking the torture code > in samba and the smbclient and to see what order to punch the > lpcfg_stuff to make smbcli_full_connection() not fail. > > On Tue, Jan 20, 2015 at 2:33 PM, Jens Axboe wrote: >> On 01/19/2015 08:31 AM, Jens Axboe wrote: >>> >>> I didn't look at your code yet, but I'm assuming it's a self contained >>> IO engine. So we should be able to make that work, by only linking the >>> engine itself against libsmbclient. But sheesh, what a pain in the butt, >>> why can't we just all be friends. >> >> >> I pulled it in for testing, and came up with this patch [1]. If you don't do >> anything, it'll build cifs into fio as before. If you add --cifs-external to >> the configure arguments, it'll build cifs.so as an externally loadable >> module. You'd then use: >> >> ioengine=/path/to/cifs.so >> >> to use that module. I did not add an install target, I'll leave that to >> distros... >> >> Let me know how that works for you. And let me know how far along you are >> with the cifs engine, I'd like to pull it in. >> >> http://git.kernel.dk/?p=fio.git;a=shortlog;h=refs/heads/cifs >> >> [1] >> http://git.kernel.dk/?p=fio.git;a=commit;h=c2c05e33b753ae686e24b43d1034d0c474203729 >> >> -- >> Jens Axboe >> > > > > -- > Milosz Tanski > CTO > 16 East 34th Street, 15th floor > New York, NY 10016 > > p: 646-253-9055 > e: milosz@adfin.com -- Milosz Tanski CTO 16 East 34th Street, 15th floor New York, NY 10016 p: 646-253-9055 e: milosz@adfin.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps Date: Tue, 20 Jan 2015 16:26:46 -0700 Message-ID: <54BEE436.4020205@kernel.dk> References: <20150115223157.GB25884@quack.suse.cz> <20150116165506.GA10856@samba2> <20150119071218.GA9747@jeremy-HP> <1421652849.2080.20.camel@HansenPartnership.com> <54BD234F.3060203@kernel.dk> <54BEAD82.3070501@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: James Bottomley , Jeremy Allison , Volker Lendecke , Jan Kara , Christoph Hellwig , linux-mm@kvack.org, "linux-fsdevel@vger.kernel.org" , lsf-pc@lists.linux-foundation.org To: Milosz Tanski Return-path: Received: from mail-pd0-f179.google.com ([209.85.192.179]:42242 "EHLO mail-pd0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751841AbbATX0b (ORCPT