Idea for optimize an OSD rebuild

Idea for optimize an OSD rebuild

[Vincent Godin]

When you replace a failed osd, it has to recover all of its pgs and so
it is pretty busy. Is it possible to tell this OSD to not become
primary for any of its already synchronized pgs till every pgs (of the
OSD) have recovered ? It should accelerate the rebuild process because
this OSD won't have to serve client's read requests but just sync the
writes request from the OSDs owning the primary pgs and recover. Is
there something wrong with this idea ?

Re: Idea for optimize an OSD rebuild

[huang jun]

Current ceph implement backfill mechanism,
let say, pg A primary is on the new osd,
but it will not serve client io during backfilling, bc there is a
replica  act as temp primary,
so it can recover fast,
after backfill finished, the osd's role will turn to primary, and
accept client io.

2017-03-30 17:47 GMT+08:00 Vincent Godin <vince.mlist@gmail.com>:
> When you replace a failed osd, it has to recover all of its pgs and so
> it is pretty busy. Is it possible to tell this OSD to not become
> primary for any of its already synchronized pgs till every pgs (of the
> OSD) have recovered ? It should accelerate the rebuild process because
> this OSD won't have to serve client's read requests but just sync the
> writes request from the OSDs owning the primary pgs and recover. Is
> there something wrong with this idea ?
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Thank you!
HuangJun

Re: Idea for optimize an OSD rebuild

[Sage Weil]

On Thu, 30 Mar 2017, Vincent Godin wrote:
> When you replace a failed osd, it has to recover all of its pgs and so
> it is pretty busy. Is it possible to tell this OSD to not become
> primary for any of its already synchronized pgs till every pgs (of the
> OSD) have recovered ? It should accelerate the rebuild process because
> this OSD won't have to serve client's read requests but just sync the
> writes request from the OSDs owning the primary pgs and recover. Is
> there something wrong with this idea ?

This is a great idea, and it's on the list of things ceph-mgr should be 
doing to balance load across the cluster.  I've added a card for this
at

        https://trello.com/b/ugTc2QFH/ceph-backlog

("mgr: primary-affinity away from busy osds").

sage

Re: Idea for optimize an OSD rebuild

[David Casier]

Hi Vincent,

You can define the affinity on the osd :

ceph osd primary-affinity osd.<ID> 0


________________________________________________________

Cordialement,

David CASIER

________________________________________________________


2017-03-30 11:47 GMT+02:00 Vincent Godin <vince.mlist@gmail.com>:
>
> When you replace a failed osd, it has to recover all of its pgs and so
> it is pretty busy. Is it possible to tell this OSD to not become
> primary for any of its already synchronized pgs till every pgs (of the
> OSD) have recovered ? It should accelerate the rebuild process because
> this OSD won't have to serve client's read requests but just sync the
> writes request from the OSDs owning the primary pgs and recover. Is
> there something wrong with this idea ?
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html