From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F7FC33CB1 for ; Thu, 16 Jan 2020 14:29:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4422221582 for ; Thu, 16 Jan 2020 14:29:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726343AbgAPO3o (ORCPT ); Thu, 16 Jan 2020 09:29:44 -0500 Received: from mx2.suse.de ([195.135.220.15]:39414 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726088AbgAPO3o (ORCPT ); Thu, 16 Jan 2020 09:29:44 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 4DF8DB370; Thu, 16 Jan 2020 14:29:42 +0000 (UTC) Received: by ds.suse.cz (Postfix, from userid 10065) id C2099DA791; Thu, 16 Jan 2020 15:29:28 +0100 (CET) Date: Thu, 16 Jan 2020 15:29:28 +0100 From: David Sterba To: Qu Wenruo Cc: linux-btrfs@vger.kernel.org Subject: Re: [PATCH] btrfs: statfs: Don't reset f_bavail if we're over committing metadata space Message-ID: <20200116142928.GX3929@twin.jikos.cz> Reply-To: dsterba@suse.cz Mail-Followup-To: dsterba@suse.cz, Qu Wenruo , linux-btrfs@vger.kernel.org References: <20200115034128.32889-1-wqu@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200115034128.32889-1-wqu@suse.com> User-Agent: Mutt/1.5.23.1-rc1 (2014-03-12) Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Wed, Jan 15, 2020 at 11:41:28AM +0800, Qu Wenruo wrote: > [BUG] > When there are a lot of metadata space reserved, e.g. after balancing a > data block with many extents, vanilla df would report 0 available space. > > [CAUSE] > btrfs_statfs() would report 0 available space if its metadata space is > exhausted. > And the calculation is based on currently reserved space vs on-disk > available space, with a small headroom as buffer. > When there is not enough headroom, btrfs_statfs() will report 0 > available space. > > The problem is, since commit ef1317a1b9a3 ("btrfs: do not allow > reservations if we have pending tickets"), we allow btrfs to over commit > metadata space, as long as we have enough space to allocate new metadata > chunks. > > This makes old calculation unreliable and report false 0 available space. > > [FIX] > Don't do such naive check anymore for btrfs_statfs(). > Also remove the comment about "0 available space when metadata is > exhausted". This is intentional and was added to prevent a situation where 'df' reports available space but exhausted metadata don't allow to create new inode. If it gets removed you are trading one bug for another. With the changed logic in the referenced commit, the metadata exhaustion is more likely but it's also temporary. The overcommit and overestimated reservations make it hard if not impossible to do any accurate calculation in statfs/df. From the usability side, there are 2 options: a) return 0 free, while it's still possible to eg. create files b) return >0 free, but no new file can be created The user report I got was for b) so that's what the guesswork fixes and does a). The idea behind that is that there's really low space, but with the overreservation caused by balance it's not. I don't see a good way out of that which could be solved inside statfs, it only interprets the numbers in the best way under circumstances. We don't have exact reservation, don't have a delta of the reserved-requested (to check how much the reservation is off).