archive mirror
 help / color / mirror / Atom feed
From: Roger Heflin <>
To: LVM general discussion and development <>
Cc: Malin Bruland <>
Subject: Re: [linux-lvm] lvconvert --uncache takes hours
Date: Thu, 2 Mar 2023 12:33:28 -0600	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Thu, Mar 2, 2023 at 11:44 AM Gionatan Danti <> wrote:
> Il 2023-03-02 01:51 Roger Heflin ha scritto:
> > A spinning raid6 array is slow on writes (see raid6  write penalty).
> > Because of that the array can only do about 100 write operattions/sec.
> True. But does flushing cached data really proceed in random LBA order
> (as seen by HDDs), rather than trying to coalesce writes in linear
> fashion?
It is a 100G cache over 16TB, so even if it flushes in order the may
not be that close to each other (1 in 160).

Also if pieces are decided and added to the cached then the cache is
not in order on the ssd and proper coalescing would require reading
the entire cache and sorting the 3,000,000 location entries before
starting the de-stage.  And that complication of a de-stage is likely
not been coded yet if I was just guessing, the de-stage starts at the
beginning and continues to the end of the cache.

Even coded though, if the you have enough blocks cached and if the
blocks spread say one or 2 on each track it would break down to having
to write a tiny bit on each track with seeks between mostly breaking
down to the time required to simply read/write  the HD end to end.  At
150MB/sec (should be about the platter speed) that would take 3.5

> > If the disk is doing other work then it only has the extra capacity so
> > it could destage slower.
> >
> > A lot depends on how big each chunk is.     The lvmcache indicates the
> > smallest chunksize is 32k.
> >
> > 100G / 32k = 3 million, and at 100seeks/sec that comes to at least an
> > hour.
> You are off an order of magnitude: 3 millions IOP at 100 IOPs means
> ~30000s, so about 9 hours.

Right, I did the calc in my head and screwed it up.  I thought it
should have been higher but did not re-check it.
> > Lvm bookkeeping has to also be written to the spinning disks I would
> > think, so 2 hours if the array were idle.
> >
> > Throw in a 50% baseload on the disks and you get 4 hours.
> >
> > Hours is reasonable.
> If flushing happens in random disk order, than yes, you are bound to
> wait several hours indeed.

linux-lvm mailing list
read the LVM HOW-TO at

  reply	other threads:[~2023-03-02 18:33 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-01 22:44 [linux-lvm] lvconvert --uncache takes hours Roy Sigurd Karlsbakk
2023-03-01 22:55 ` Demi Marie Obenour
2023-03-02  0:51 ` Roger Heflin
2023-03-02  8:33   ` Roy Sigurd Karlsbakk
2023-03-02 11:27     ` Roger Heflin
2023-03-02 17:34   ` Gionatan Danti
2023-03-02 18:33     ` Roger Heflin [this message]
2023-03-02 20:47       ` Gionatan Danti

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).