Linux-mtd Archive on lore.kernel.org
 help / color / Atom feed
From: Richard Weinberger <richard.weinberger@gmail.com>
To: Ben Schroeder <klowd92@gmail.com>
Cc: linux-mtd@lists.infradead.org
Subject: Re: Available space loss due to fragmentation?
Date: Wed, 10 Jul 2019 11:34:07 +0200
Message-ID: <CAFLxGvy=iwwUbS8O1xMAtMZYibzQ+vbft1ZVpg9erR=S1c8s2g@mail.gmail.com> (raw)
In-Reply-To: <CAMk-x8NpDDjqtJQ0WqqRcggQxBp3uu7QbQET0m019LoG9OSkvA@mail.gmail.com>

Ben,

On Wed, Jul 10, 2019 at 5:32 AM Ben Schroeder <klowd92@gmail.com> wrote:
> Why do I see a loss of space when rewriting the same file?

Please see my answer below.

> Can I use an upgrade scheme with file binary diff as mentioned above -
> One that would run correctly with low available space?

If the filesystem is full and all nodes are already packed, it can be
a challenge.

> Can I use an upgrade scheme with UBI volume binary diff?

Yes, you can alter a dynamic volume as you wish. But keep NAND odds on mind.
So you need to replace whole LEBs.

> Sorry for the long mail, I have not found much information about fragmentation
> and space loss in UBIFS. Let me know if I forgot any relevant details.

I think the root cause of the problem you see is how NAND works.
On NAND we write always full pages. So if you ask UBIFS to change one byte
of a file or change meta data, it has to waste a full page.

Luckily Linux is a modern operating system with a write-cache and upon
write-back UBIFS can pack nodes (UBIFS data nodes, inode nodes, etc...) together
and wastes less space.
But it wastes still a significant amount of space if userspace forces
it to persist data.
i.e. by using fsync()/fdatasync().
If UBIFS runs out of space the garbage collector will rewrite nodes
and pack them tightly
together.

So, if you have a pre-created UBIFS, nodes are already packed and your
update mechanism
might force UBIFS to faster than the garbage collector can pack nodes.

With that information in mind, do your other questions resolve?

-- 
Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

  reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-10  3:32 Ben Schroeder
2019-07-10  9:34 ` Richard Weinberger [this message]
2019-07-10 15:18   ` Ben Schroeder
2019-07-11 10:16     ` Richard Weinberger
2019-07-11 15:53       ` Ben Schroeder
2019-07-11 20:41         ` Richard Weinberger

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFLxGvy=iwwUbS8O1xMAtMZYibzQ+vbft1ZVpg9erR=S1c8s2g@mail.gmail.com' \
    --to=richard.weinberger@gmail.com \
    --cc=klowd92@gmail.com \
    --cc=linux-mtd@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mtd Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mtd/0 linux-mtd/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mtd linux-mtd/ https://lore.kernel.org/linux-mtd \
		linux-mtd@lists.infradead.org linux-mtd@archiver.kernel.org
	public-inbox-index linux-mtd


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-mtd


AGPL code for this site: git clone https://public-inbox.org/ public-inbox