From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gerd Hoffmann Subject: Re: [Qemu-devel] [RFC] Replace posix-aio with custom thread pool Date: Thu, 11 Dec 2008 16:24:37 +0100 Message-ID: <494130B5.2080800@redhat.com> References: <1228512061-25398-1-git-send-email-aliguori@us.ibm.com> <493E941D.4000608@redhat.com> <493E965E.5050701@us.ibm.com> <20081210164401.GF18814@random.random> <493FFAB6.2000106@codemonkey.ws> <493FFC8E.9080802@redhat.com> <49400F69.8080707@codemonkey.ws> <20081210190810.GG18814@random.random> <20081211131222.GA14908@random.random> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Anthony Liguori , kvm-devel , qemu-devel@nongnu.org To: Andrea Arcangeli Return-path: Received: from mx2.redhat.com ([66.187.237.31]:38183 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755589AbYLKPYp (ORCPT ); Thu, 11 Dec 2008 10:24:45 -0500 In-Reply-To: <20081211131222.GA14908@random.random> Sender: kvm-owner@vger.kernel.org List-ID: Andrea Arcangeli wrote: > My current feeling is that this user thread aio thing will never > satisfy enterprise usage and kernel aio is mandatory in my view. Well, linux kernel aio has its share of problems too: * Anthony mentioned it may block on certain circumstances (forgot which ones), and you can't figure beforehand to turn off aio then. * It can't handle block allocation. Kernel handles that by doing such writes synchronously via VFS layer (instead of the separate aio code paths). Leads to horrible performance and bug reports such as "installs on sparse files are very slow". * support for vectored aio isn't that old. IIRC it was added somewhen around 2.6.20 (newer that current suse/redhat enterprise versions). Which IMHO means you can't expect it being present unconditionally. > And we should concentrate on kernel aio and get rid > of threads when host OS is linux. Threads will be there anyway for kvm smp. > Has anybody a patch implementing kernel aio that I can plug into the > dma zerocopy api? I'm not so sure clone aio is worth maintaining > inside qemu instead of evolving glibc Well, wait for glibc isn't going to fly. glibc waits for posix, and posix waits for a reference implementation (which will not be glibc). > and kernel with preadv/pwritev With that in place you don't need kernel aio any more, then you can really do it in userspace with threads. But that probably would be linux-only ^W^W^W ahem: http://www.daemon-systems.org/man/preadv.2.html cheers, Gerd From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LAnPC-0001tB-TT for qemu-devel@nongnu.org; Thu, 11 Dec 2008 10:24:46 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LAnPB-0001st-5v for qemu-devel@nongnu.org; Thu, 11 Dec 2008 10:24:46 -0500 Received: from [199.232.76.173] (port=47358 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LAnPB-0001sq-2R for qemu-devel@nongnu.org; Thu, 11 Dec 2008 10:24:45 -0500 Received: from mx2.redhat.com ([66.187.237.31]:58944) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1LAnPA-0000qe-KQ for qemu-devel@nongnu.org; Thu, 11 Dec 2008 10:24:44 -0500 Message-ID: <494130B5.2080800@redhat.com> Date: Thu, 11 Dec 2008 16:24:37 +0100 From: Gerd Hoffmann MIME-Version: 1.0 Subject: Re: [Qemu-devel] [RFC] Replace posix-aio with custom thread pool References: <1228512061-25398-1-git-send-email-aliguori@us.ibm.com> <493E941D.4000608@redhat.com> <493E965E.5050701@us.ibm.com> <20081210164401.GF18814@random.random> <493FFAB6.2000106@codemonkey.ws> <493FFC8E.9080802@redhat.com> <49400F69.8080707@codemonkey.ws> <20081210190810.GG18814@random.random> <20081211131222.GA14908@random.random> In-Reply-To: <20081211131222.GA14908@random.random> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andrea Arcangeli Cc: kvm-devel , qemu-devel@nongnu.org Andrea Arcangeli wrote: > My current feeling is that this user thread aio thing will never > satisfy enterprise usage and kernel aio is mandatory in my view. Well, linux kernel aio has its share of problems too: * Anthony mentioned it may block on certain circumstances (forgot which ones), and you can't figure beforehand to turn off aio then. * It can't handle block allocation. Kernel handles that by doing such writes synchronously via VFS layer (instead of the separate aio code paths). Leads to horrible performance and bug reports such as "installs on sparse files are very slow". * support for vectored aio isn't that old. IIRC it was added somewhen around 2.6.20 (newer that current suse/redhat enterprise versions). Which IMHO means you can't expect it being present unconditionally. > And we should concentrate on kernel aio and get rid > of threads when host OS is linux. Threads will be there anyway for kvm smp. > Has anybody a patch implementing kernel aio that I can plug into the > dma zerocopy api? I'm not so sure clone aio is worth maintaining > inside qemu instead of evolving glibc Well, wait for glibc isn't going to fly. glibc waits for posix, and posix waits for a reference implementation (which will not be glibc). > and kernel with preadv/pwritev With that in place you don't need kernel aio any more, then you can really do it in userspace with threads. But that probably would be linux-only ^W^W^W ahem: http://www.daemon-systems.org/man/preadv.2.html cheers, Gerd