Linux-mtd Archive on lore.kernel.org
 help / color / Atom feed
From: Ben Schroeder <klowd92@gmail.com>
To: linux-mtd@lists.infradead.org
Subject: Available space loss due to fragmentation?
Date: Wed, 10 Jul 2019 06:32:16 +0300
Message-ID: <CAMk-x8NpDDjqtJQ0WqqRcggQxBp3uu7QbQET0m019LoG9OSkvA@mail.gmail.com> (raw)

Hi everyone,
I have an issue of loss of available space after file rewriting -

Setup:
3 UBI Volumes (25mb static, 25mb static, 15mb dynamic) on a single MTD device.
A/B partition scheme for upgrades with the 25mb volumes.
A/B partitions have around 400kb space available each, and cannot
expand beyond it.

Issue:
Partition 'A' is main RootFs, 'B' partition mounted on /mnt/ to be upgraded.
When attempting to upgrade the backup partition, by rewriting the files
according to a binary diff, I run out of available space on the volume.

The issue can be simplified:
Copying a 2mb file out of the partition, and then back to overwrite the original
may cause loss of 100kb of space.
Rewriting files with minor changes, even if the new file is smaller than the
original, can result in loss of available disk space.

I base this not only on the info from: df -h
But also run sync, unmount and mount again, and reboot.
All show a loss of available space.
I am aware that df -h does not show an accurate value, however, copying
and modifying files will fail due to no available disk space error!

Some writes will continue to succeed even when available space is 4kb.
But many writes will fail when available space is low.

I am working on a new format of a UBIFS image.
I suspect that the original UBI volume files are alligned perfectly,
and once I rewrite files wit binary diffs, the files become fragmented,
and a loss of available space occurs, even though the files remain the
same or smaller.

However, I am not familiar enough with the UBI internals to be certain.

Why do I see a loss of space when rewriting the same file?
Can I use an upgrade scheme with file binary diff as mentioned above -
One that would run correctly with low available space?
Can I use an upgrade scheme with UBI volume binary diff?

Sorry for the long mail, I have not found much information about fragmentation
and space loss in UBIFS. Let me know if I forgot any relevant details.

Thanks in advance.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

             reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-10  3:32 Ben Schroeder [this message]
2019-07-10  9:34 ` Richard Weinberger
2019-07-10 15:18   ` Ben Schroeder
2019-07-11 10:16     ` Richard Weinberger
2019-07-11 15:53       ` Ben Schroeder
2019-07-11 20:41         ` Richard Weinberger

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAMk-x8NpDDjqtJQ0WqqRcggQxBp3uu7QbQET0m019LoG9OSkvA@mail.gmail.com \
    --to=klowd92@gmail.com \
    --cc=linux-mtd@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mtd Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mtd/0 linux-mtd/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mtd linux-mtd/ https://lore.kernel.org/linux-mtd \
		linux-mtd@lists.infradead.org linux-mtd@archiver.kernel.org
	public-inbox-index linux-mtd


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-mtd


AGPL code for this site: git clone https://public-inbox.org/ public-inbox