Re: [linux-lvm] lvconvert --uncache takes hours

[Roger Heflin <rogerheflin@gmail.com>]

On Thu, Mar 2, 2023 at 11:44 AM Gionatan Danti <g.danti@assyoma.it> wrote:
>
> Il 2023-03-02 01:51 Roger Heflin ha scritto:
> > A spinning raid6 array is slow on writes (see raid6  write penalty).
> > Because of that the array can only do about 100 write operattions/sec.
>
> True. But does flushing cached data really proceed in random LBA order
> (as seen by HDDs), rather than trying to coalesce writes in linear
> fashion?
>
It is a 100G cache over 16TB, so even if it flushes in order the may
not be that close to each other (1 in 160).

Also if pieces are decided and added to the cached then the cache is
not in order on the ssd and proper coalescing would require reading
the entire cache and sorting the 3,000,000 location entries before
starting the de-stage.  And that complication of a de-stage is likely
not been coded yet if I was just guessing, the de-stage starts at the
beginning and continues to the end of the cache.

Even coded though, if the you have enough blocks cached and if the
blocks spread say one or 2 on each track it would break down to having
to write a tiny bit on each track with seeks between mostly breaking
down to the time required to simply read/write  the HD end to end.  At
150MB/sec (should be about the platter speed) that would take 3.5
hours.


> > If the disk is doing other work then it only has the extra capacity so
> > it could destage slower.
> >
> > A lot depends on how big each chunk is.     The lvmcache indicates the
> > smallest chunksize is 32k.
> >
> > 100G / 32k = 3 million, and at 100seeks/sec that comes to at least an
> > hour.
>
> You are off an order of magnitude: 3 millions IOP at 100 IOPs means
> ~30000s, so about 9 hours.

Right, I did the calc in my head and screwed it up.  I thought it
should have been higher but did not re-check it.
>
> > Lvm bookkeeping has to also be written to the spinning disks I would
> > think, so 2 hours if the array were idle.
> >
> > Throw in a 50% baseload on the disks and you get 4 hours.
> >
> > Hours is reasonable.
>
> If flushing happens in random disk order, than yes, you are bound to
> wait several hours indeed.
>

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/