Client Message Priority Optimazation

Client Message Priority Optimazation

[Ning Yao]

Hi, all

Currently,  we can modify osd_client_op_priority to assign different
clients' ops with different priority such like we can assign high
priority for OLTP and assign low priority for OLAP. However, there are
some consideration as below:

1) it seems OLTP's client op still can be blocked by OLAP's sub_ops
since sub_ops use the CEPH_MSG_PRIO_DEFAULT.  So should we consider
sub_op should inherit the message's priority from client Ops (if
client ops do not give priority  explicitly, use CEPH_MSG_PRIO_DEFAULT
by default), does this make sense?

2) secondly, reply message is assigned with
priority(CEPH_MSG_PRIO_HIGH), but there is no restriction for client
Ops' priority (use can set 210), which will lead to blocked for reply
message. So should we think change those kind of message to highest
priority(CEPH_MSG_PRIO_HIGHEST). Currently, it seems no ops use
CEPH_MSG_PRIO_HIGHEST.

3) I think the kick recovery ops should inherit the client ops priority

4) Is that possible to add test cases to verify whether it works
properly as expected in ceph-qa-suite as Sam mentioned before? Any
guidelines?


Regards
Ning Yao

Re: Client Message Priority Optimazation

[Xinze Chi (信泽)]

Creating special pool for OLAP looks more simple to me.

Because there are several queue in osd. It may be hard to process
in-flight op based on priority.   I think it is hard to achieve what
you need based on priority.

2016-12-03 17:21 GMT+08:00 Ning Yao <zay11022@gmail.com>:
> Hi, all
>
> Currently,  we can modify osd_client_op_priority to assign different
> clients' ops with different priority such like we can assign high
> priority for OLTP and assign low priority for OLAP. However, there are
> some consideration as below:
>
> 1) it seems OLTP's client op still can be blocked by OLAP's sub_ops
> since sub_ops use the CEPH_MSG_PRIO_DEFAULT.  So should we consider
> sub_op should inherit the message's priority from client Ops (if
> client ops do not give priority  explicitly, use CEPH_MSG_PRIO_DEFAULT
> by default), does this make sense?
>
> 2) secondly, reply message is assigned with
> priority(CEPH_MSG_PRIO_HIGH), but there is no restriction for client
> Ops' priority (use can set 210), which will lead to blocked for reply
> message. So should we think change those kind of message to highest
> priority(CEPH_MSG_PRIO_HIGHEST). Currently, it seems no ops use
> CEPH_MSG_PRIO_HIGHEST.
>
> 3) I think the kick recovery ops should inherit the client ops priority
>
> 4) Is that possible to add test cases to verify whether it works
> properly as expected in ceph-qa-suite as Sam mentioned before? Any
> guidelines?
>
>
> Regards
> Ning Yao
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Regards,
Xinze Chi

Re: Client Message Priority Optimazation

[Ning Yao]

Yeah, I think you mean different pools with using different osds. But
sometimes, we want to deal with different application in the same
architecture or hardwares because of the limitation of company's
budget.

I think we can shift the disscussion to Ceph QoS user stories proposed
by Sage. And consider it at the same time. I think it is quite
complicated based on the current op queue strategy.
Regards
Ning Yao


2016-12-03 17:41 GMT+08:00 Xinze Chi (信泽) <xmdxcxz@gmail.com>:
> Creating special pool for OLAP looks more simple to me.
>
> Because there are several queue in osd. It may be hard to process
> in-flight op based on priority.   I think it is hard to achieve what
> you need based on priority.
>
> 2016-12-03 17:21 GMT+08:00 Ning Yao <zay11022@gmail.com>:
>> Hi, all
>>
>> Currently,  we can modify osd_client_op_priority to assign different
>> clients' ops with different priority such like we can assign high
>> priority for OLTP and assign low priority for OLAP. However, there are
>> some consideration as below:
>>
>> 1) it seems OLTP's client op still can be blocked by OLAP's sub_ops
>> since sub_ops use the CEPH_MSG_PRIO_DEFAULT.  So should we consider
>> sub_op should inherit the message's priority from client Ops (if
>> client ops do not give priority  explicitly, use CEPH_MSG_PRIO_DEFAULT
>> by default), does this make sense?
>>
>> 2) secondly, reply message is assigned with
>> priority(CEPH_MSG_PRIO_HIGH), but there is no restriction for client
>> Ops' priority (use can set 210), which will lead to blocked for reply
>> message. So should we think change those kind of message to highest
>> priority(CEPH_MSG_PRIO_HIGHEST). Currently, it seems no ops use
>> CEPH_MSG_PRIO_HIGHEST.
>>
>> 3) I think the kick recovery ops should inherit the client ops priority
>>
>> 4) Is that possible to add test cases to verify whether it works
>> properly as expected in ceph-qa-suite as Sam mentioned before? Any
>> guidelines?
>>
>>
>> Regards
>> Ning Yao
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Regards,
> Xinze Chi