Available space loss due to fragmentation?

From: Ben Schroeder <klowd92@gmail.com>
To: linux-mtd@lists.infradead.org
Subject: Available space loss due to fragmentation?
Date: Wed, 10 Jul 2019 06:32:16 +0300	[thread overview]
Message-ID: <CAMk-x8NpDDjqtJQ0WqqRcggQxBp3uu7QbQET0m019LoG9OSkvA@mail.gmail.com> (raw)

Hi everyone,
I have an issue of loss of available space after file rewriting -

Setup:
3 UBI Volumes (25mb static, 25mb static, 15mb dynamic) on a single MTD device.
A/B partition scheme for upgrades with the 25mb volumes.
A/B partitions have around 400kb space available each, and cannot
expand beyond it.

Issue:
Partition 'A' is main RootFs, 'B' partition mounted on /mnt/ to be upgraded.
When attempting to upgrade the backup partition, by rewriting the files
according to a binary diff, I run out of available space on the volume.

The issue can be simplified:
Copying a 2mb file out of the partition, and then back to overwrite the original
may cause loss of 100kb of space.
Rewriting files with minor changes, even if the new file is smaller than the
original, can result in loss of available disk space.

I base this not only on the info from: df -h
But also run sync, unmount and mount again, and reboot.
All show a loss of available space.
I am aware that df -h does not show an accurate value, however, copying
and modifying files will fail due to no available disk space error!

Some writes will continue to succeed even when available space is 4kb.
But many writes will fail when available space is low.

I am working on a new format of a UBIFS image.
I suspect that the original UBI volume files are alligned perfectly,
and once I rewrite files wit binary diffs, the files become fragmented,
and a loss of available space occurs, even though the files remain the
same or smaller.

However, I am not familiar enough with the UBI internals to be certain.

Why do I see a loss of space when rewriting the same file?
Can I use an upgrade scheme with file binary diff as mentioned above -
One that would run correctly with low available space?
Can I use an upgrade scheme with UBI volume binary diff?

Sorry for the long mail, I have not found much information about fragmentation
and space loss in UBIFS. Let me know if I forgot any relevant details.

Thanks in advance.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/