From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f67.google.com ([209.85.218.67]:36397 "EHLO mail-oi0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756269AbcJNViu (ORCPT ); Fri, 14 Oct 2016 17:38:50 -0400 Received: by mail-oi0-f67.google.com with SMTP id e12so8304062oib.3 for ; Fri, 14 Oct 2016 14:38:49 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20161014195544.GZ21290@hungrycats.org> References: <20161011160601.GI7683@carfax.org.uk> <3da9a459-c63b-570c-5b42-c7186b3a74fd@cn.fujitsu.com> <20161012043718.GW21290@hungrycats.org> <37578baa-556b-d3f7-45bd-10843124dea1@cn.fujitsu.com> <20161012171936.GD26140@hungrycats.org> <16a02312-a74c-6e2d-08ae-7fa479823341@cn.fujitsu.com> <20161013210346.GY21290@hungrycats.org> <20161014195544.GZ21290@hungrycats.org> From: Chris Murphy Date: Fri, 14 Oct 2016 15:38:48 -0600 Message-ID: Subject: Re: RAID system with adaption to changed number of disks To: Zygo Blaxell Cc: Chris Murphy , Qu Wenruo , Hugo Mills , "linux-btrfs@vger.kernel.org" , Austin Hemmelgarn Content-Type: text/plain; charset=UTF-8 Sender: linux-btrfs-owner@vger.kernel.org List-ID: On Fri, Oct 14, 2016 at 1:55 PM, Zygo Blaxell wrote: > >> And how common is RMW for metadata operations? > > RMW in metadata is the norm. It happens on nearly all commits--the only > exception seems to be when both ends of a commit write happen to land > on stripe boundaries accidentally, which is less than 1% of the time on > 3 disks. In the interest of due diligence, and the fact I can't confirm or deny this myself from reading the code (although I do see many comments involving RMW in the code), I must ask Qu if he can corroborate this. Because basically means btrfs raid56 is not better than md raid56 - by design. It has nothing to do with bugs. This is substantially worse than the scrub->wrong parity bug. Does it make sense to proscribe raid5 profile for metadata? As in, disallow -m raid5 at mkfs time? Maybe recommend raid1. Even raid6 seems like it could be specious - yes there are two copies but if there is constant RMW, then there's no CoW and we're not really protected that well with all of these overwrites, statistically speaking. Basically you have to have a setup where there's no chance of torn or misdirected writes, and no corruptions, in which case Btrfs checksums aren't really helpful, you're using it for other reasons (snapshots and what not). Really seriously the CoW part of Btrfs being violated by all of this RMW to me sounds like it reduces the pros of Btrfs. -- Chris Murphy