Lowering the weight to move PGs out

Lowering the weight to move PGs out

[Loic Dachary]

Hi Sage,

The most time consuming part of the optimization algorithm[1] comes from lowering the weight of the most over filled item. We cannot know exactly which value will push the desired number of PGs out. The values influenced by the weight are calculated during mapping[2] and derived from the parameters.

It would be useful find a way to expose all ln values (i.e. log(hash(PGid,r,item)) without the weight applied. A function behaving like map() but returning all straws from bucket_straw2_choose instead of chosing one based on the weights.

With the actual values influenced by the weights, we can calculate how much to lower the weight of a given item so that it does not win. Instead of trying small increments, we calculate the exact value.

What do you think ?

Cheers

[1] An algorithm to fix uneven CRUSH distributions in Ceph http://dachary.org/?p=4055
[2] https://github.com/ceph/ceph/blob/master/src/crush/mapper.c#L323
-- 
Loïc Dachary, Artisan Logiciel Libre

Re: Lowering the weight to move PGs out

[Loic Dachary]

Hi Sage,

There is a draft implementation of the rebalancing algorithm at http://libcrush.org/main/libcrush/merge_requests/32/diffs and it hopefully shows the benefit for exposing straws.

Cheers

On 05/28/2017 10:17 AM, Loic Dachary wrote:
> Hi Sage,
> 
> The most time consuming part of the optimization algorithm[1] comes from lowering the weight of the most over filled item. We cannot know exactly which value will push the desired number of PGs out. The values influenced by the weight are calculated during mapping[2] and derived from the parameters.
> 
> It would be useful find a way to expose all ln values (i.e. log(hash(PGid,r,item)) without the weight applied. A function behaving like map() but returning all straws from bucket_straw2_choose instead of chosing one based on the weights.
> 
> With the actual values influenced by the weights, we can calculate how much to lower the weight of a given item so that it does not win. Instead of trying small increments, we calculate the exact value.
> 
> What do you think ?
> 
> Cheers
> 
> [1] An algorithm to fix uneven CRUSH distributions in Ceph http://dachary.org/?p=4055
> [2] https://github.com/ceph/ceph/blob/master/src/crush/mapper.c#L323
> 

-- 
Loïc Dachary, Artisan Logiciel Libre