Regarding cache tier understanding

Regarding cache tier understanding

[Somnath Roy]

[-- Attachment #1.1: Type: text/plain, Size: 2440 bytes --]

Hi Sage,
I went through the tiering agent code base and here is my understanding on the agent behavior. Please let me know if that is correct.


1.       Agent will be always in idle state if target_max_bytes or target_max_objects not set on the pool irrespective of other tiering params set in the pool. dirty_micro and full_micro will not be calculated if those two params are zero which is by default I guess.

2.       Now, flush will be activated if dirty_micro is > flush_target. My understanding is, once it is activated it will iterate through all the dirty objects and flush all the dirty objects which is > cache_min_flush_age. Am I right ?

3.       For the eviction, if full_micro > 1000000 , the mode set as TierAgentState::EVICT_MODE_FULL and all the clean objects are flushed.

4.       if (full_micro > evict_target), the mode is set as TierAgentState::EVICT_MODE_SOME. In this scenario evict_effort is calculated and based on hit_set and temp calculation some clean objects are evicted. My question is , can we quantify this value ? For ex, if the target_full_ratio = 50%, once the eviction is triggered, what %objects will be evicted ?

5.       Also, why we are multiplying with 1000000 always ? Is it because in the histogram it is positioned as 0..1000000 ?

6.       I saw the cache_min_evict_age is not been used anywhere, am I missing anything ?

7.       The cache_min_flush_age will only be applicable if the flush is triggered after crossing the dirty_threshold, right ? If dirty_threshold is not breached, the flush age param is never checked.

I need to understand the behavior so that we can test the Cache tiering properly.
Is there any behavior I am missing here which you want us to test out ?

Thanks & Regards
Somnath

________________________________

PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).


[-- Attachment #1.2: Type: text/html, Size: 12321 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: Regarding cache tier understanding

[Sage Weil]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3030 bytes --]

On Thu, 7 Aug 2014, Somnath Roy wrote:
> Hi Sage,
> 
> I went through the tiering agent code base and here is my understanding on
> the agent behavior. Please let me know if that is correct.
> 
>  
> 
> 1.       Agent will be always in idle state if target_max_bytes or
> target_max_objects not set on the pool irrespective of other tiering params
> set in the pool. dirty_micro and full_micro will not be calculated if those
> two params are zero which is by default I guess.

Yeah

> 2.       Now, flush will be activated if dirty_micro is > flush_target. My
> understanding is, once it is activated it will iterate through all the dirty
> objects and flush all the dirty objects which is > cache_min_flush_age. Am I
> right ?

Yeah

> 3.       For the eviction, if full_micro > 1000000 , the mode set as
> TierAgentState::EVICT_MODE_FULL and all the clean objects are flushed.

Yeah

> 4.       if (full_micro > evict_target), the mode is set as
> TierAgentState::EVICT_MODE_SOME. In this scenario evict_effort is calculated
> and based on hit_set and temp calculation some clean objects are evicted. My
> question is , can we quantify this value ? For ex, if the target_full_ratio
> = 50%, once the eviction is triggered, what %objects will be evicted ?

The effort is calculated based on how far between target_full and 100% we 
are.  This is mapped onto the estimation of atime.  We generate a 
histogram of age for the objects we have examined, so after the agent has 
run a bit we'll have a reasonable idea how the current object's age 
compares to others and can decide whether this is older or newer than the 
others; based on that we decide what to do.

> 5.       Also, why we are multiplying with 1000000 always ? Is it because in
> the histogram it is positioned as 0..1000000 ?

Yeah, and we use "micro" units throughout to avoid doing any floating 
point.

> 6.       I saw the cache_min_evict_age is not been used anywhere, am I
> missing anything ?

It's possible.  The _min_ params are a bit dangerous because they can 
potentially confuse the cache agent (e.g., what if *all* objects are under 
the min?  How/when do we decide to ignore the min, or, how/when do we 
give up trying to find an older object?).

> 7.       The cache_min_flush_age will only be applicable if the flush is
> triggered after crossing the dirty_threshold, right ? If dirty_threshold is
> not breached, the flush age param is never checked.

Right.

> I need to understand the behavior so that we can test the Cache tiering
> properly.
> 
> Is there any behavior I am missing here which you want us to test out ?

There is a new behavior just merged into master that controls our decision 
to promote data into the cache on read; see maybe_handle_cache().  That's 
not strictly agent behavior per se.  Also, there is now a readforward mode 
that doesn't promote on read ever, based on our discussion about the 
performance of flash on read.

Hope this helps!
sage

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: Regarding cache tier understanding

[Somnath Roy]

Thanks Sage, this helps !

Regards
Somnath

-----Original Message-----
From: Sage Weil [mailto:sweil-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org]
Sent: Thursday, August 07, 2014 7:12 AM
To: Somnath Roy
Cc: ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org; Anirban Ray; Allen Samuels
Subject: Re: Regarding cache tier understanding

On Thu, 7 Aug 2014, Somnath Roy wrote:
> Hi Sage,
>
> I went through the tiering agent code base and here is my
> understanding on the agent behavior. Please let me know if that is correct.
>
>
>
> 1.       Agent will be always in idle state if target_max_bytes or
> target_max_objects not set on the pool irrespective of other tiering
> params set in the pool. dirty_micro and full_micro will not be
> calculated if those two params are zero which is by default I guess.

Yeah

> 2.       Now, flush will be activated if dirty_micro is >
> flush_target. My understanding is, once it is activated it will
> iterate through all the dirty objects and flush all the dirty objects
> which is > cache_min_flush_age. Am I right ?

Yeah

> 3.       For the eviction, if full_micro > 1000000 , the mode set as
> TierAgentState::EVICT_MODE_FULL and all the clean objects are flushed.

Yeah

> 4.       if (full_micro > evict_target), the mode is set as
> TierAgentState::EVICT_MODE_SOME. In this scenario evict_effort is
> calculated and based on hit_set and temp calculation some clean
> objects are evicted. My question is , can we quantify this value ? For
> ex, if the target_full_ratio = 50%, once the eviction is triggered, what %objects will be evicted ?

The effort is calculated based on how far between target_full and 100% we are.  This is mapped onto the estimation of atime.  We generate a histogram of age for the objects we have examined, so after the agent has run a bit we'll have a reasonable idea how the current object's age compares to others and can decide whether this is older or newer than the others; based on that we decide what to do.

> 5.       Also, why we are multiplying with 1000000 always ? Is it
> because in the histogram it is positioned as 0..1000000 ?

Yeah, and we use "micro" units throughout to avoid doing any floating point.

> 6.       I saw the cache_min_evict_age is not been used anywhere, am I
> missing anything ?

It's possible.  The _min_ params are a bit dangerous because they can potentially confuse the cache agent (e.g., what if *all* objects are under the min?  How/when do we decide to ignore the min, or, how/when do we give up trying to find an older object?).

> 7.       The cache_min_flush_age will only be applicable if the flush
> is triggered after crossing the dirty_threshold, right ? If
> dirty_threshold is not breached, the flush age param is never checked.

Right.

> I need to understand the behavior so that we can test the Cache
> tiering properly.
>
> Is there any behavior I am missing here which you want us to test out ?

There is a new behavior just merged into master that controls our decision to promote data into the cache on read; see maybe_handle_cache().  That's not strictly agent behavior per se.  Also, there is now a readforward mode that doesn't promote on read ever, based on our discussion about the performance of flash on read.

Hope this helps!
sage

________________________________

PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).