From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [Qemu-devel] [RFC] Replace posix-aio with custom thread pool Date: Wed, 10 Dec 2008 11:21:58 -0600 Message-ID: <493FFAB6.2000106@codemonkey.ws> References: <1228512061-25398-1-git-send-email-aliguori@us.ibm.com> <493E941D.4000608@redhat.com> <493E965E.5050701@us.ibm.com> <20081210164401.GF18814@random.random> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: qemu-devel@nongnu.org, Gerd Hoffmann , kvm-devel To: Andrea Arcangeli Return-path: Received: from yx-out-2324.google.com ([74.125.44.28]:24143 "EHLO yx-out-2324.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754916AbYLJRWE (ORCPT ); Wed, 10 Dec 2008 12:22:04 -0500 Received: by yx-out-2324.google.com with SMTP id 8so283644yxm.1 for ; Wed, 10 Dec 2008 09:22:02 -0800 (PST) In-Reply-To: <20081210164401.GF18814@random.random> Sender: kvm-owner@vger.kernel.org List-ID: Andrea Arcangeli wrote: > On Tue, Dec 09, 2008 at 10:01:34AM -0600, Anthony Liguori wrote: > >> Yes, that's what I'm going to do before committing it. >> > > I've been hacking on this too, just to push out a full tested patchset > without the _em badness... problem is if we use more than one thread, > there's a thread race between lseek and writev, pread/pwrite don't > have the issue as they store the pos on the kernel stack, and they > don't pass through the shared file->f_pos. We'd really need > preadv/pwritev... > > To solve this in userland without kernel aio we'd need to open (not > just dup) Why not just dup? I've implemented this and it seems to work. > the file in each thread, then the file->f_pos will become > thread local and we can cache the last lseek value and avoid the lseek > syscall for contiguous I/O. Or we need to reduce the number of threads > to 1 per fd (screwing seeking I/O). kernel aio wouldn't have this > trouble and a single fd/file would be enough, but that would only work > on linux. > I've also done a preadv/pwritev implementation in userspace using linux-aio. My current plan is to finish the refactoring, then test out each implementation (dup() + lseek + readv vs. linux-aio) to see which one performs better. If they're equal, dup() wins out because it's more portable. Regards, Anthony Liguori > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1LASlB-0005z4-Cg for qemu-devel@nongnu.org; Wed, 10 Dec 2008 12:22:05 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1LASl9-0005wr-Bh for qemu-devel@nongnu.org; Wed, 10 Dec 2008 12:22:04 -0500 Received: from [199.232.76.173] (port=47406 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1LASl9-0005wf-6r for qemu-devel@nongnu.org; Wed, 10 Dec 2008 12:22:03 -0500 Received: from yx-out-1718.google.com ([74.125.44.154]:53634) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1LASl8-000335-Sf for qemu-devel@nongnu.org; Wed, 10 Dec 2008 12:22:03 -0500 Received: by yx-out-1718.google.com with SMTP id 3so265788yxi.82 for ; Wed, 10 Dec 2008 09:22:02 -0800 (PST) Message-ID: <493FFAB6.2000106@codemonkey.ws> Date: Wed, 10 Dec 2008 11:21:58 -0600 From: Anthony Liguori MIME-Version: 1.0 Subject: Re: [Qemu-devel] [RFC] Replace posix-aio with custom thread pool References: <1228512061-25398-1-git-send-email-aliguori@us.ibm.com> <493E941D.4000608@redhat.com> <493E965E.5050701@us.ibm.com> <20081210164401.GF18814@random.random> In-Reply-To: <20081210164401.GF18814@random.random> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Andrea Arcangeli Cc: qemu-devel@nongnu.org, kvm-devel , Gerd Hoffmann Andrea Arcangeli wrote: > On Tue, Dec 09, 2008 at 10:01:34AM -0600, Anthony Liguori wrote: > >> Yes, that's what I'm going to do before committing it. >> > > I've been hacking on this too, just to push out a full tested patchset > without the _em badness... problem is if we use more than one thread, > there's a thread race between lseek and writev, pread/pwrite don't > have the issue as they store the pos on the kernel stack, and they > don't pass through the shared file->f_pos. We'd really need > preadv/pwritev... > > To solve this in userland without kernel aio we'd need to open (not > just dup) Why not just dup? I've implemented this and it seems to work. > the file in each thread, then the file->f_pos will become > thread local and we can cache the last lseek value and avoid the lseek > syscall for contiguous I/O. Or we need to reduce the number of threads > to 1 per fd (screwing seeking I/O). kernel aio wouldn't have this > trouble and a single fd/file would be enough, but that would only work > on linux. > I've also done a preadv/pwritev implementation in userspace using linux-aio. My current plan is to finish the refactoring, then test out each implementation (dup() + lseek + readv vs. linux-aio) to see which one performs better. If they're equal, dup() wins out because it's more portable. Regards, Anthony Liguori > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >