Re: ENOSPC on file system with nearly empty 12TB drive

From: Chris Murphy <lists@colorremedies.com>
To: Pete <pete@petezilla.co.uk>
Cc: Chris Murphy <lists@colorremedies.com>,
	Btrfs BTRFS <linux-btrfs@vger.kernel.org>
Subject: Re: ENOSPC on file system with nearly empty 12TB drive
Date: Sat, 15 Jan 2022 13:38:14 -0700	[thread overview]
Message-ID: <CAJCQCtS8DDGXOjZdzvkaMSbVyuG-x1Z5o_fO_u8rOGtE2zKSfA@mail.gmail.com> (raw)
In-Reply-To: <ed95c0e2-3029-45d0-c039-a275037d8bf4@petezilla.co.uk>

On Fri, Jan 14, 2022 at 4:09 PM Pete <pete@petezilla.co.uk> wrote:

> Any suggestions?  Just be patient, and hope the balance finishes without
> ENOSPC?  Go for the remove again.  I'd like to remove a 4TB drive if I
> can without adding a 6th HD to the system.  Still don't understand why I
> might need more than one practically empty drive for raid1?

When bg profile is raid1, any time the file system wants to add
another block group, it must create a chunk on 2 devices at the same
time for it to succeed or else you get ENOSPC. The question is why two
chunks can't be created given all the unallocated space you have even
without the empty drive.

Ordinarily you want to avoid doing metadata balance. Balancing data is
ok, it amounts to defragmenting free space, and returning excess into
unallocated space which can then be used to create any type of block
group.

>
> Or, given that I've got another 12TB drive on the way, just abandon this
> file system (apart from reading any relevant data) and create a new file
> system as single, and migrate the current 12TB disk over later on to go
> back to raid1?
>
> I'm wondering if part of the problem is a 3.6TB disk image that is
> sitting on the file system (fixing someone else's external drive).
> Deleting that might speed things up, but since I don't think that drive
> is fully backed up I'd rather keep the disk image until I am sure that
> all is well.  (I don't need the backup lecture...)

I don't think the large image file is the problem.

In my opinion, you've hit a bug. There's plenty of unallocated space
on multiple drives. I think what's going on is an old bug that might
not be fixed yet where metadata overcommit is allowed to overestimate
due to the single large empty drive with a ton of space on it. But
since the overcommit can't be fulfilled on at least two drives, you
get ENOSPC even though it's not actually trying to create that many
block groups at once.

So what I suggest doing is removing the mostly empty device, and only
add devices in pairs when using raid1.

-- 
Chris Murphy