archive mirror
 help / color / mirror / Atom feed
From: Dave Chinner <>
To: Chris Dunlop <>
Cc: Eric Sandeen <>,
Subject: Re: XFS fallocate implementation incorrectly reports ENOSPC
Date: Fri, 27 Aug 2021 15:49:56 +1000	[thread overview]
Message-ID: <20210827054956.GP3657114@dread.disaster.area> (raw)
In-Reply-To: <>

On Fri, Aug 27, 2021 at 12:55:39PM +1000, Chris Dunlop wrote:
> On Fri, Aug 27, 2021 at 06:56:35AM +1000, Chris Dunlop wrote:
> > On Thu, Aug 26, 2021 at 10:05:00AM -0500, Eric Sandeen wrote:
> > > On 8/25/21 9:06 PM, Chris Dunlop wrote:
> > > > 
> > > > fallocate -l 1GB image.img
> > > > mkfs.xfs -f image.img
> > > > mkdir mnt
> > > > mount -o loop ./image.img mnt
> > > > fallocate -o 0 -l 700mb mnt/image.img
> > > > fallocate -o 0 -l 700mb mnt/image.img
> > > > 
> > > > Why does the second fallocate fail with ENOSPC, and is that considered an XFS bug?
> > > 
> > > Interesting.  Off the top of my head, I assume that xfs is not looking at
> > > current file space usage when deciding how much is needed to satisfy the
> > > fallocate request.  While filesystems can return ENOSPC at any time for
> > > any reason, this does seem a bit suboptimal.
> > 
> > Yes, I would have thought the second fallocate should be a noop.
> On further reflection, "filesystems can return ENOSPC at any time" is
> certainly something apps need to be prepared for (and in this case, it's
> doing the right thing, by logging the error and aborting), but it's not
> really a "not a bug" excuse for the filesystem in all circumstances (or this
> one?), is it? E.g. a write(fd, buf, 1) returning ENOSPC on an fresh
> filesystem would be considered a bug, no?

Sure, but the fallocate case here is different. You're asking to
preallocate up to 700MB of space on a filesystem that only has 300MB
of space free. Up front, without knowing anything about the layout
of the file we might need to allocate 700MB of space into, there's a
very good chance that we'll get ENOSPC partially through the

The real problem with preallocation failing part way through due to
overcommit of space is that we can't go back an undo the
allocation(s) made by fallocate because when we get ENOSPC we have
lost all the state of the previous allocations made. If fallocate is
filling holes between unwritten extents already in the file, then we
have no way of knowing where the holes we filled were and hence
cannot reliably free the space we've allocated before ENOSPC was

Hence if we allow the fallocate to go ahead and preallocate space
until we hit ENOSPC, we still end up returning to userspace with
ENOSPC, but we've also consumed all the remaining space in the

So there's a very good argument for simply rejecting any attempt to
preallocate space that has the possibility of over-committing space
and hence hitting ENOSPC part way through. Given that we spend a lot
of effort in XFS to avoid over-committing resources so that ENOSPC
is reliable and not prone to deadlocks, the choice to make fallocate
avoid a potential over-commit is at least internally consistent with
the XFS ENOSPC architecture.

IOWs, either behaviour could be considered a "bug" because it is
sub-optimal behaviour, but at some point you've got to choose what
is the least worst behaviour and run with it.

> ...or maybe your "suboptimal" was entirely tongue in cheek?
> > > > Background: I'm chasing a mysterious ENOSPC error on an XFS
> > > > filesystem with way more space than the app should be asking
> > > > for. There are no quotas on the fs. Unfortunately it's a third
> > > > party app and I can't tell what sequence is producing the error,
> > > > but this fallocate issue is a possibility.

More likely speculative preallocation is causing this than
fallocate. However, we've had a background worker that cleans up
speculative prealloc before reporting ENOSPC for a while now - what
kernel version are seeing this on?

Also, it might not even be data allocation that is the issue - if
the filesystem is full and free space is fragmented, you could be
getting ENOSPC because inodes cannot be allocated. In which case,
the output of xfs-info would be useful so we can see if sparse inode
clusters are enabled or not....


Dave Chinner

  reply	other threads:[~2021-08-27  5:50 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-26  2:06 XFS fallocate implementation incorrectly reports ENOSPC Chris Dunlop
2021-08-26 15:05 ` Eric Sandeen
2021-08-26 20:56   ` Chris Dunlop
2021-08-27  2:55     ` Chris Dunlop
2021-08-27  5:49       ` Dave Chinner [this message]
2021-08-27  6:53         ` Chris Dunlop
2021-08-27 22:03           ` Dave Chinner
2021-08-28  0:21             ` Mysterious ENOSPC [was: XFS fallocate implementation incorrectly reports ENOSPC] Chris Dunlop
2021-08-28  3:58               ` Chris Dunlop
2021-08-29 22:04                 ` Dave Chinner
2021-08-30  4:21                   ` Darrick J. Wong
2021-08-30  7:40                     ` Chris Dunlop
2021-08-30  7:37                   ` Mysterious ENOSPC Chris Dunlop
2021-09-02  1:42                     ` Dave Chinner
2021-09-17  6:07                       ` Chris Dunlop

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210827054956.GP3657114@dread.disaster.area \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).