"No space left on device" issue

* "No space left on device" issue
@ 2022-02-17 19:52 Manish Adkar
  2022-02-17 22:34 ` Dave Chinner
  0 siblings, 1 reply; 2+ messages in thread
From: Manish Adkar @ 2022-02-17 19:52 UTC (permalink / raw)
  To: linux-xfs; +Cc: Erez Zadok, Yifei Liu

Hello,

We have a 16 MB XFS test partition with both plenty of free space
and inodes (419 out of 424), but we’re still getting the error "No
space left on device" when trying to create new files (After the
free space reaches 184320 bytes). It can then only edit and append
any existing files. (There are good reasons why we want to avoid a
larger partition.)

Mount options:

# mount |grep xfs
/dev/ram1 on /mnt/test-xfs type xfs (rw,noatime,attr2,inode64,
logbufs=8,logbsize=32k,noquota)

Here, the filesystem is already mounted with the inode64 option.

Used disk space:

# df /dev/ram1
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/ram1          11500 11320       180  99% /mnt/test-xfs

Used inodes:

# df -i /dev/ram1
Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/ram1         424     5   419    2% /mnt/test-xfs

xfs info:

# xfs_info /dev/ram1
meta-data=/dev/ram1      isize=512    agcount=1, agsize=4096 blks
         =               sectsz=4096  attr=2, projid32bit=1
         =               crc=1        finobt=1, sparse=1, rmapbt=0
         =               reflink=1    bigtime=0
data     =               bsize=4096   blocks=4096, imaxpct=25
         =               sunit=0      swidth=0 blks
naming   =version 2      bsize=4096   ascii-ci=0, ftype=1
log      =internal log   bsize=4096   blocks=1221, version=2
         =               sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none           extsz=4096   blocks=0, rtextents=0

In the below “freesp” output, we can see that there are plenty of
contiguous blocks for inode cluster allocation and hence the disk
is not fragmented.

# xfs_db -r -c "freesp -s -a 0" /dev/ram1
   from      to extents  blocks    pct
      1       1       4       4   1.44
      4       7       1       6   2.16
    256     511       1     268  96.40
total free extents 6
total free blocks 278
average free extent size 46.3333

I came across an email thread from the linux-xfs mailing list that
described a similar issue. It mentioned, inodes are allocated in
contiguous chunks of 64. Here, in this case, 1 inode takes 512 bytes
of space and 1 block has 4096 bytes capacity. Hence there would be 8
inodes per block (4096 / 512). Now, to allocate 64 inodes, 8 blocks
would be needed (64 / 8). Looking at the above “freesp” output, we
can see there are 8 contiguous blocks available so the issue must not
be a fragmentation one. But the thread suggests, the allocation has
an alignment requirement, and here, the blocks must be aligned to an
8 block boundary. To look for that,

# xfs_db -r -c "freesp -s -a 0 -A 8" /dev/ram1
   from      to extents  blocks    pct
      1       1       1       1 100.00
total free extents 1
total free blocks 1
average free extent size 1

Here we can see that there is just one single block that is aligned
to the 8 block boundary. Hence, this could be a fragmentation issue.
My concern is that, is my understanding and calculations
(64 / (4096 / 512)) correct about the 8-block boundary? I also tried
this

# xfs_db -r -c "freesp -s -a 0 -A 32" /dev/ram1
   from      to extents  blocks    pct
total free extents 0
total free blocks 0
average free extent size -nan

And this also suggests that there are no such required blocks.

I also saw the suggestion given in this thread to avoid this issue.
It mentions that enabling “sparse” inode allocations will help
resolve this issue. But here “sparse” inode allocation is enabled by
default. So probably there is another resolution to this problem.
Can you help?

^ permalink raw reply	[flat|nested] 2+ messages in thread