From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Nelson Subject: Re: Poor read performance in KVM Date: Fri, 20 Jul 2012 11:53:01 -0500 Message-ID: <50098CED.4020609@inktank.com> References: <5002C215.108@bashkirtsev.com> <5003B1CC.4060909@inktank.com> <50064DCD.8040904@bashkirtsev.com> <5006D5FB.8030700@inktank.com> <50080D9D.8010306@bashkirtsev.com> <50085518.80507@inktank.com> <500984AC.9030104@bashkirtsev.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from mail-gh0-f174.google.com ([209.85.160.174]:41700 "EHLO mail-gh0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752894Ab2GTQxE (ORCPT ); Fri, 20 Jul 2012 12:53:04 -0400 Received: by ghrr11 with SMTP id r11so4188382ghr.19 for ; Fri, 20 Jul 2012 09:53:03 -0700 (PDT) In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Tommi Virtanen Cc: Vladimir Bashkirtsev , Josh Durgin , ceph-devel On 7/20/12 11:42 AM, Tommi Virtanen wrote: > On Fri, Jul 20, 2012 at 9:17 AM, Vladimir Bashkirtsev > wrote: >> not running. So I ended up rebooting hosts and that's where fun begin: btrfs >> has failed to umount , on boot up it spit out "btrfs: free space inode >> generation (0) did not match free space cache generation (177431)". I have >> not started ceph and made an attempt to umount and umount just froze. >> Another reboot: same stuff. I have rebooted second host and it came back >> with the same error. So in effect I was unable to mount btrfs and read it: >> no wonder that ceph was unable to run. Actually according to mons ceph was > > The btrfs developers tend to be good about bug reports that severe -- > I think you should email that mailing list and ask if that sounds like > known bug, and ask what information you should capture if it happens > again (assuming the workload is complex enough that you can't easily > capture/reproduce all of that). > >> But it leaves me with very final question: should we rely on btrfs at this >> point given it is having such major faults? What if I will use well tested >> by time ext4? > > You might want to try xfs. We hear/see problems with all three, but > xfs currently seems to have the best long-term performance and > reliability. > > I'm not sure if anyone's run detailed tests with ext4 after the > xattrs-in-leveldb feature; before that, we ran into fs limitations. > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > I ran some limited ext4 tests a little while back. Performance was on-par with btrfs on a fresh filesystem and generally better than xfs. I was not able to test performance on an aged filesystem at that time. I did run into a situation at one point where ext4 broke rather spectacularly and the system needed to be rebooted and fscks done on the filesystems. Other than that it was stable during the duration of the tests. Mark