From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FORGED_MUA_MOZILLA,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0507C00454 for ; Mon, 9 Dec 2019 19:27:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 749FF20726 for ; Mon, 9 Dec 2019 19:27:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=urbackup.org header.i=@urbackup.org header.b="NyEUqkm3"; dkim=pass (1024-bit key) header.d=amazonses.com header.i=@amazonses.com header.b="b0tP2jbF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726691AbfLIT1I (ORCPT ); Mon, 9 Dec 2019 14:27:08 -0500 Received: from a4-5.smtp-out.eu-west-1.amazonses.com ([54.240.4.5]:37716 "EHLO a4-5.smtp-out.eu-west-1.amazonses.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727022AbfLIT05 (ORCPT ); Mon, 9 Dec 2019 14:26:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ob2ngmaigrjtzxgmrxn2h6b3gszyqty3; d=urbackup.org; t=1575919615; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=5hAGiIr1Wjve3zDg/iE0S3d8A2M/Xsr6sYT+7o+1f70=; b=NyEUqkm3wNqlp4+nTMkFiKWp3E+5sHdneGEd9iDXM3c2P4vUhRUNA3Du0kCJso8n P+xdOlO5c5hgZ9TYtUv6Z97DkUWZKK2gPpD5SftHtscuvEvC7/+GAusBBoNLcLz+w9b Wi2gF92LmUYEOtZ9sbBM8WDMT/w87dBEsr6p545E= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ihchhvubuqgjsxyuhssfvqohv7z3u4hn; d=amazonses.com; t=1575919615; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=5hAGiIr1Wjve3zDg/iE0S3d8A2M/Xsr6sYT+7o+1f70=; b=b0tP2jbFF8pXfO8ccDiNkHfWRxm5J+4KC6vw8iKPA/CLZZtZFWm2JJfPteHG4IyS bKkWy4cufPJ0MwZ6dl9d6UiLiKXhdmrF8GtriwOjFFvBPwG4gCXovCa3VlyD5yxzDaE 0gJZBFhw5tyPVzkdhLt3wIFEpv7xfD8+iiXf4PWo= Subject: Re: df shows no available space in 5.4.1 To: Qu Wenruo , "linux-btrfs@vger.kernel.org" References: <0102016edd1b0184-848d9b6d-6b80-4ce3-8428-e472a224e554-000000@eu-west-1.amazonses.com> <784074e1-667a-a2c7-5b47-7cbe36f5fdf5@gmx.com> <0102016eec056406-8dc0180d-5a2d-44e8-9ae2-f02573e62203-000000@eu-west-1.amazonses.com> From: Martin Raiber Message-ID: <0102016eec20fb02-4931c649-abbf-46c6-bb4b-eca248df3213-000000@eu-west-1.amazonses.com> Date: Mon, 9 Dec 2019 19:26:54 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <0102016eec056406-8dc0180d-5a2d-44e8-9ae2-f02573e62203-000000@eu-west-1.amazonses.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-SES-Outgoing: 2019.12.09-54.240.4.5 Feedback-ID: 1.eu-west-1.zKMZH6MF2g3oUhhjaE2f3oQ8IBjABPbvixQzV8APwT0=:AmazonSES Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On 09.12.2019 19:56 Martin Raiber wrote: > On 07.12.2019 08:28 Qu Wenruo wrote: >> On 2019/12/7 上午5:26, Martin Raiber wrote: >>> Hi, >>> >>> with kernel 5.4.1 I have the problem that df shows 100% space used. I >>> can still write to the btrfs volume, but my software looks at the >>> available space and starts deleting stuff if statfs() says there is a >>> low amount of available space. >> If the bug still happens, mind to try the snippet to see why this happened? >> >> You will need to: >> - Apply the patch to your kernel code >> - Recompile the kernel or btrfs module >> So this needs some experience in kernel compile. >> - Reboot to newly compiled kernel or load the debug btrfs module >> >> Thanks, >> Qu >> >> diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c >> index 23aa630f04c9..cf34c05b16d7 100644 >> --- a/fs/btrfs/relocation.c >> +++ b/fs/btrfs/relocation.c >> @@ -523,7 +523,8 @@ static int should_ignore_root(struct btrfs_root *root) >> { >> struct btrfs_root *reloc_root; >> >> - if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state)) >> + if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state) || >> + test_bit(BTRFS_ROOT_DEAD_RELOC_TREE, &root->state)) >> return 0; >> >> reloc_root = root->reloc_root; >> diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c >> index f452a94abdc3..c2b70d97a63b 100644 >> --- a/fs/btrfs/super.c >> +++ b/fs/btrfs/super.c >> @@ -2064,6 +2064,8 @@ static int btrfs_statfs(struct dentry *dentry, >> struct kstatfs *buf) >> found->disk_used; >> } >> >> + pr_info("%s: found type=0x%llx disk_used=%llu factor=%d\n", >> + __func__, found->flags, found->disk_used, factor); >> total_used += found->disk_used; >> } >> >> @@ -2071,6 +2073,8 @@ static int btrfs_statfs(struct dentry *dentry, >> struct kstatfs *buf) >> >> buf->f_blocks = div_u64(btrfs_super_total_bytes(disk_super), >> factor); >> buf->f_blocks >>= bits; >> + pr_info("%s: super_total_bytes=%llu total_used=%llu >> factor=%d\n", __func__, >> + btrfs_super_total_bytes(disk_super), total_used, factor); >> buf->f_bfree = buf->f_blocks - (div_u64(total_used, factor) >> >> bits); >> >> /* Account global block reserve as used, it's in logical size >> already */ >> > Applied. It's currently 100% used directly after reboot, and I am > getting this log output: > > [...] > [  241.245150] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  241.904824] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  241.904824] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  241.904824] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  241.904824] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  241.976082] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  241.976082] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  241.976082] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  241.976082] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  241.976083] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  241.976083] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  241.976083] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  241.976083] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  242.245301] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  242.245301] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  242.245301] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  242.245301] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  242.904977] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  242.904977] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  242.904977] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  242.904977] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  242.976105] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  242.976105] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  242.976105] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  242.976105] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  242.977777] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  242.977777] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  242.977777] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  242.977777] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  243.245041] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  243.245041] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  243.245041] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.245041] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  243.252079] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1 > [  243.252079] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1 > [  243.252079] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.252079] btrfs_statfs: super_total_bytes=49999998976 > total_used=30983798784 factor=1 > [  243.252080] btrfs_statfs: found type=0x1 disk_used=30026579968 factor=1 > [  243.252080] btrfs_statfs: found type=0x4 disk_used=957202432 factor=1 > [  243.252080] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.252080] btrfs_statfs: super_total_bytes=49999998976 > total_used=30983798784 factor=1 > [  243.904085] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  243.904086] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  243.904086] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.904086] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  243.976096] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  243.976096] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  243.976096] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.976096] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  243.976257] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  243.976257] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  243.976257] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  243.976258] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  244.081959] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1 > [  244.081959] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1 > [  244.081959] btrfs_statfs: found type=0x2 disk_used=98304 factor=1 > [  244.081959] btrfs_statfs: super_total_bytes=8133881348096 > total_used=670555975680 factor=1 > [  244.084110] btrfs_statfs: found type=0x1 disk_used=665652932608 factor=1 > [  244.084110] btrfs_statfs: found type=0x4 disk_used=4902944768 factor=1 > [  244.084110] btrfs_statfs: found type=0x2 disk_used=98304 factor=1 > [  244.084110] btrfs_statfs: super_total_bytes=8133881348096 > total_used=670555975680 factor=1 > [  244.245470] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  244.245470] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  244.245470] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  244.245470] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  244.904170] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  244.904170] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  244.904170] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  244.904170] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  244.977624] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  244.977624] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  244.977624] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  244.977624] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  244.977387] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  244.977387] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  244.977387] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  244.977387] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 > [  245.244089] btrfs_statfs: found type=0x1 disk_used=93464006656 factor=1 > [  245.244089] btrfs_statfs: found type=0x4 disk_used=314818560 factor=1 > [  245.244089] btrfs_statfs: found type=0x2 disk_used=16384 factor=1 > [  245.244089] btrfs_statfs: super_total_bytes=128835387392 > total_used=93778841600 factor=1 It went back to 9% afterwards (without reboot, sync() perhaps?). I hope it didn't go back while the above was logged. I'll keep an eye on it. >>> # df -h >>> Filesystem                                            Size  Used Avail >>> Use% Mounted on >>> ... >>> /dev/loop0                                            7.4T  623G     0 >>> 100% /media/backup >>> ... >>> >>> statfs("/media/backup", {f_type=BTRFS_SUPER_MAGIC, f_bsize=4096, >>> f_blocks=1985810876, f_bfree=1822074245, f_bavail=0, f_files=0, >>> f_ffree=0, f_fsid={val=[3667078581, 2813298474]}, f_namelen=255, >>> f_frsize=4096, f_flags=ST_VALID|ST_NOATIME}) = 0 >>> >>> # btrfs fi usage /media/backup >>> Overall: >>>     Device size:                   7.40TiB >>>     Device allocated:            671.02GiB >>>     Device unallocated:            6.74TiB >>>     Device missing:                  0.00B >>>     Used:                        622.49GiB >>>     Free (estimated):              6.79TiB      (min: 6.79TiB) >>>     Data ratio:                       1.00 >>>     Metadata ratio:                   1.00 >>>     Global reserve:              512.00MiB      (used: 0.00B) >>> >>> Data,single: Size:666.01GiB, Used:617.95GiB >>>    /dev/loop0    666.01GiB >>> >>> Metadata,single: Size:5.01GiB, Used:4.54GiB >>>    /dev/loop0      5.01GiB >>> >>> System,single: Size:4.00MiB, Used:96.00KiB >>>    /dev/loop0      4.00MiB >>> >>> Unallocated: >>>    /dev/loop0      6.74TiB >>> >>> # btrfs fi df /media/backup >>> Data, single: total=666.01GiB, used=617.95GiB >>> System, single: total=4.00MiB, used=96.00KiB >>> Metadata, single: total=5.01GiB, used=4.54GiB >>> GlobalReserve, single: total=512.00MiB, used=0.00B >>> >>> # mount >>> >>> ... >>> /dev/loop0 on /media/backup type btrfs >>> (rw,noatime,nossd,discard,space_cache=v2,enospc_debug,skip_balance,commit=86400,subvolid=5,subvol=/) >>> ... >>> >>> (I remounted with enospc_debug and the available space did not change...) >>> >>> Regards, >>> Martin Raiber >>>