Re: Best way to add caching to a new raid setup.

[Roman Mamedov <rm@romanrm.net>]

On Fri, 28 Aug 2020 22:08:22 -0500
"R. Ramesh" <rramesh@verizon.net> wrote:

> I do not know how SSD caching is implemented. I assumed it will be 
> somewhat similar to memory cache (L2 vs L3 vs L4 etc). I am hoping that 
> with SSD caching, reads/writes to disk will be larger in size and 
> sequential within a file (similar to cache line fill in memory cache 
> which results in memory bursts that are efficient). I thought that is 
> what SSD caching will do to disk reads/writes. I assumed, once reads 
> (ahead) and writes (assuming writeback cache) buffers data sufficiently 
> in the SSD, all reads/writes will be to SSD with periodic well organized 
> large transfers to disk. If I am wrong here then I do not see any point 
> in SSD as a cache. My aim is not to optimize by cache hits, but optimize 
> by preventing disks from thrashing back and forth seeking after every 
> block read. I suppose Linux (memory) buffer cache alleviates some of 
> that. I was hoping SSD will provide next level. If not, I am off in my 
> understanding of SSD as a disk cache.

Just try it, as I said before with LVM it is easy to remove if it doesn't work
out. You can always go to the manual copying method or whatnot, but first why
not check if the automatic caching solution might be "good enough" for your
needs.

Yes it usually tries to avoid caching long sequential reads or writes, but
there's also quite a bit of other load on the FS, i.e. metadata. I found that
browsing directories and especially mounting the filesystem had a great
benefit from caching.

You are correct that it will try to increase performance via writeback
caching, however with LVM that needs to be enabled explicitly:
https://www.systutorials.com/docs/linux/man/7-lvmcache/#lbAK
And of course a failure of that cache SSD will mean losing some data, even if
the main array is RAID. Perhaps should consider a RAID of SSDs for cache in
that case then.

-- 
With respect,
Roman