From: CoolCold <coolthecold@gmail.com>
To: linux-lvm@redhat.com
Subject: [linux-lvm] dm-cache / lvmcache - no cleanup for dirty data in writeback/cleaner mode
Date: Tue, 7 Jun 2022 00:33:17 +0700 [thread overview]
Message-ID: <CAGqmV7o+nFVUkbSpt_oRVbHshv-yWn56Rxk6b7=rpZC7fiLrdg@mail.gmail.com> (raw)
Good day!
Main issue as I see it: dirty data size grows, cached data size
declines, no automagic flushing happens (or much slower than growing
up)
Sub issue: cleaner policy flushes data back to origin device very slowly.
LVM version: 2.03.07(2) (2019-11-30)
Kernel 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022
x86_64 x86_64 x86_64 GNU/Linux
Sizes: 2TB origin/700GB cache device
[cachedata_cpool] vg0 Cwi---C--- 700.00g
99.99 21.21 72.70
[cachedata_cpool_cdata] vg0 Cwi-ao---- 700.00g
[cachedata_cpool_cmeta] vg0 ewi-ao---- 20.00m
Let's start from cleaner mode, as it what is happening on server right now:
root@TasteWP2:~# lvs -o+cache_policy,cache_settings
LV VG Attr LSize Pool Origin Data%
Meta% Move Log Cpy%Sync Convert CachePolicy CacheSettings
data vg0 Cwi-aoC--- 2.00t [cachedata_cpool] [data_corig] 99.99
21.21 72.70 cleaner
migration_threshold=65536
and Cpy%Sync moving down very slowly - over 4 hours, it moved from
72.74% to 72.70% only.
The origin device (a couple of HDDs in mdadm RAID1) is almost idling,
5-15% busy based on iostat.
graphical representation from collectd/rrd - https://snipboard.io/WNdGhz.jpg
What I've tried to do: play around setting migration_threshhold, but
see no changes
lvchange --cachesettings 'migration_threshold=65536' vg0/data
lvchange --cachesettings 'migration_threshold=8192' vg0/data
---
I think this subissue leads to the main issue with dirty growing up,
not leaving space for "real" data.
My very limited knowledge, checking through history on github
https://github.com/torvalds/linux/commits/master/drivers/md/dm-cache-target.c
back to 2019 (time of kernel release)
have not provided any hints on fixes/changes added here, so I'm bit
lost. Help is appreciated.
--
Best regards,
[COOLCOLD-RIPN]
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
reply other threads:[~2022-06-06 17:33 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAGqmV7o+nFVUkbSpt_oRVbHshv-yWn56Rxk6b7=rpZC7fiLrdg@mail.gmail.com' \
--to=coolthecold@gmail.com \
--cc=linux-lvm@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).