Re: Best way to add caching to a new raid setup.

[Ram Ramesh <rramesh2400@gmail.com>]

On 8/29/20 5:36 PM, Drew wrote:
> I know what you and Wols are talking about and I think it's actually
> two separate things. Wol's is referring to traditional read caching
> where it only benefits if you are reading the same thing over and over
> again, cache hits. For streaming it won't help as you'll never hit the
> cache.
>
> What you are talking about is a write cache, something I have seen
> implemented before. Basically the idea is for writes to hit the SSD's
> first, the SSD acting as a cache or buffer between the filesystem and
> the slower RAID array. To the end process they're just writing to a
> disk, they don't see the SSD buffer/cache. QNAP implements this in
> their NAS chassis, just not sure what the exact implementation is in
> their case.
>
> On Fri, Aug 28, 2020 at 9:14 PM R. Ramesh <rramesh@verizon.net> wrote:
>> On 8/28/20 7:01 PM, Roger Heflin wrote:
>>> Something I would suggest, I have found improves my mythtv experience
>>> is:  Get a big enough SSD to hold 12-18 hours of the recording or
>>> whatever you do daily, and setup the recordings to go to the SSD.    i
>>> defined use the disk with the highest percentage free to be used
>>> first, and since my raid6 is always 90% plus the SSD always gets used.
>>> Then nightly I move the files from the ssd recordings directory onto
>>> the raid6 recordings directory.  This also helps when your disks start
>>> going bad and getting badblocks, the badblocks *WILL* cause mythtv to
>>> stop recording shows at random because of some prior choices the
>>> developers made (sync often, and if you get more than a few seconds
>>> behind stop recording, attempting to save some recordings).
>>>
>>> I also put daily security camera data on the ssd and copy it over to
>>> the raid6 device nightly.
>>>
>>> Using the ssd for recording much reduces the load on the slower raid6
>>> spinning disks.
>>>
>>> You would have to have a large number of people watching at the same
>>> time as the watching is relatively easy load, compared to the writes.
>>>
>>> On Fri, Aug 28, 2020 at 5:42 PM Ram Ramesh <rramesh2400@gmail.com> wrote:
>>>> On 8/28/20 5:12 PM, antlists wrote:
>>>>> On 28/08/2020 18:25, Ram Ramesh wrote:
>>>>>> I am mainly looking for IOP improvement as I want to use this RAID in
>>>>>> mythtv environment. So multiple threads will be active and I expect
>>>>>> cache to help with random access IOPs.
>>>>> ???
>>>>>
>>>>> Caching will only help in a read-after-write scenario, or a
>>>>> read-several-times scenario.
>>>>>
>>>>> I'm guessing mythtv means it's a film server? Can ALL your films (or
>>>>> at least your favourite "watch again and again" ones) fit in the
>>>>> cache? If you watch a lot of films, chances are you'll read it from
>>>>> disk (no advantage from the cache), and by the time you watch it again
>>>>> it will have been evicted so you'll have to read it again.
>>>>>
>>>>> The other time cache may be useful, is if you're recording one thing
>>>>> and watching another. That way, the writes can stall in cache as you
>>>>> prioritise reading.
>>>>>
>>>>> Think about what is actually happening at the i/o level, and will
>>>>> cache help?
>>>>>
>>>>> Cheers,
>>>>> Wol
>>>> Mythtv is a sever client DVR system. I have a client next to each of my
>>>> TVs and one backend with large disk (this will have RAID with cache). At
>>>> any time many clients will be accessing different programs and any
>>>> scheduled recording will also be going on in parallel. So you will see a
>>>> lot of seeks, but still all will be based on limited threads (I only
>>>> have 3 TVs and may be one other PC acting as a client) So lots of IOs,
>>>> mostly sequential, across small number of threads. I think most cache
>>>> algorithms should be able to benefit from random access to blocks in SSD.
>>>>
>>>> Do you see any flaws in my argument?
>>>>
>>>> Regards
>>>> Ramesh
>>>>
>> I was hoping SSD caching would do what you are suggesting without daily
>> copying. Based on Wol's comments, it does not. May be I misunderstood
>> how SSD caching works.  I will try it any way and see what happens. If
>> it does not do what I want, I will remove caching and go straight to disks.
>>
>> Ramesh
>
>
After thinking through this, I really like the idea of simply recording 
programs to SSD and move one file at a time based on some aging 
algorithms of my own. I will move files back and forth as needed during 
overnight hours creating my own caching effect. As long as I keep the 
original (renamed) and cache the ones needed with correct name, mythtv 
will find the cached copy. When mythtv complains about something 
missing, I can manually look at the renamed backup copy and make the 
corrections. Unless my thinking is badly broken, this should work.

I really wished overlay fs had a nice merge/clean feature that will 
allow us to move overlay items to underlying file system and start over 
the overlay. All I need is file level caching and not block level caching.

Ramesh