From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753277AbbE0S3A (ORCPT ); Wed, 27 May 2015 14:29:00 -0400 Received: from mail.phunq.net ([184.71.0.62]:35996 "EHLO starbase.phunq.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753221AbbE0S26 convert rfc822-to-8bit (ORCPT ); Wed, 27 May 2015 14:28:58 -0400 From: Daniel Phillips To: Mosis Tembo Cc: Pavel Machek , , , , OGAWA Hirofumi Subject: Re: Tux3 Report: How fast can we =?iso-8859-1?Q?fail=3F?= Date: Wed, 27 May 2015 11:28:50 -0700 User-Agent: Trojita/v0.5-14-g8a2496c; Qt/4.8.6; X11; Linux; Ubuntu 14.04.2 LTS MIME-Version: 1.0 Message-ID: In-Reply-To: References: <8f886f13-6550-4322-95be-93244ae61045@phunq.net> <55523C88.9080809@phunq.net> <20150526100326.GA8854@amd> Organization: tux3.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tuesday, May 26, 2015 11:41:39 PM PDT, Mosis Tembo wrote: > On Tue, May 26, 2015 at 6:03 PM, Pavel Machek wrote: > >> >>> We identified the following quality metrics for this algorithm: >>> >>> 1) Never fails to detect out of space in the front end. >>> 2) Always fills a volume to 100% before reporting out of space. >>> 3) Allows rm, rmdir and truncate even when a volume is full. > > This is definitely nonsense. You can not rm, rmdir and truncate > when the volume is full. You will need a free space on disk to perform > such operations. Do you know why? Because some extra space needs to be on the volume in order to do the atomic commit. Specifically, there must be enough extra space to keep both old and new copies of any changed metadata, plus enough space for new data or metadata. You are almost right: we can't support rm, rmdir or truncate _with atomic commit_ unless some space is available on the volume. So we keep a small reserve to handle those operations, which only those operations can access. We define the volume as "full" when only the reserve remains. The reserve is not included in "available" blocks reported to statfs, so the volume appears to be 100% full when only the reserve remains. For Tux3, that reserve is variable - about 1% of free space, declining to a minimum of 10 blocks as free space runs out. Eventually, we will reduce the minimum a bit as we develop finer control over how free space is used in very low space conditions, but 10 blocks is not bad at all. With no journal and only 10 blocks of unusable space, we do pretty well with tiny volumes. Regards, Daniel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Phillips Subject: Re: Tux3 Report: How fast can we =?iso-8859-1?Q?fail=3F?= Date: Wed, 27 May 2015 11:28:50 -0700 Message-ID: References: <8f886f13-6550-4322-95be-93244ae61045@phunq.net> <55523C88.9080809@phunq.net> <20150526100326.GA8854@amd> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Cc: linux-fsdevel@vger.kernel.org, tux3@tux3.org, OGAWA Hirofumi , linux-kernel@vger.kernel.org, Pavel Machek To: Mosis Tembo Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: tux3-bounces@phunq.net Sender: "Tux3" List-Id: linux-fsdevel.vger.kernel.org On Tuesday, May 26, 2015 11:41:39 PM PDT, Mosis Tembo wrote: > On Tue, May 26, 2015 at 6:03 PM, Pavel Machek wrote: > >> >>> We identified the following quality metrics for this algorithm: >>> >>> 1) Never fails to detect out of space in the front end. >>> 2) Always fills a volume to 100% before reporting out of space. >>> 3) Allows rm, rmdir and truncate even when a volume is full. > > This is definitely nonsense. You can not rm, rmdir and truncate > when the volume is full. You will need a free space on disk to perform > such operations. Do you know why? Because some extra space needs to be on the volume in order to do the atomic commit. Specifically, there must be enough extra space to keep both old and new copies of any changed metadata, plus enough space for new data or metadata. You are almost right: we can't support rm, rmdir or truncate _with atomic commit_ unless some space is available on the volume. So we keep a small reserve to handle those operations, which only those operations can access. We define the volume as "full" when only the reserve remains. The reserve is not included in "available" blocks reported to statfs, so the volume appears to be 100% full when only the reserve remains. For Tux3, that reserve is variable - about 1% of free space, declining to a minimum of 10 blocks as free space runs out. Eventually, we will reduce the minimum a bit as we develop finer control over how free space is used in very low space conditions, but 10 blocks is not bad at all. With no journal and only 10 blocks of unusable space, we do pretty well with tiny volumes. Regards, Daniel