All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Spray <jspray@redhat.com>
To: Sage Weil <sage@newdream.net>
Cc: Xiaoxi Chen <superdebuger@gmail.com>,
	Ceph Development <ceph-devel@vger.kernel.org>
Subject: Re: Nodown/Noout by OSD_ID?
Date: Wed, 20 Jan 2016 13:55:08 +0000	[thread overview]
Message-ID: <CALe9h7cqj+ck9-tKBgKuSx=4KcPSJnjO3O=eWSfV3yUT3J=QcQ@mail.gmail.com> (raw)
In-Reply-To: <alpine.DEB.2.11.1601200813250.4600@cpach.fuggernut.com>

On Wed, Jan 20, 2016 at 1:32 PM, Sage Weil <sage@newdream.net> wrote:
> On Wed, 20 Jan 2016, Xiaoxi Chen wrote:
>> Hi,
>>
>>      In many case we need to tag some OSD with NODOWN/NOOUT/NOUP/NOIN
>> tag, but we dont want it cluster wise as these tag may stop other OSDs
>> doing self-healthing.As a an example when an recovered OSD need to
>> catch up with the OSDMap, to prevent flipping we set
>> NODOWN/NOOUT/NOUP, but if other OSD failed by disk error, the failure
>> will be hidden and we are in the risk of lossing the data.
>>
>>      Is that reasonable to have these flag work in OSD granularity?
>> say ceph osd nodown osd.xxx?
>>      Quick look at the code seems NODOWN/NOUP is easier as we could
>> have new status bits in OSDMap
>>      /* status bits */
>> #define CEPH_OSD_EXISTS  (1<<0)
>> #define CEPH_OSD_UP      (1<<1)
>> #define CEPH_OSD_AUTOOUT (1<<2)  /* osd was automatically marked out */
>> #define CEPH_OSD_NEW     (1<<3)  /* osd is new, never marked in */
>>
>> #define CEPH_OSD_NOUP     (1<<4)  /* osd cannot be marked in */
>> #define CEPH_OSD_NODOWN     (1<<5)  /* osd cannot be marked out */
>>
>>      But for NOIN/NOOUT seems a bit struggle as IN/OUT depends on
>> weight? Any suggestion?
>
> This looks reasonable if we can sort out a good interface and suitable
> health warnings.  For example, ceph health and ceph -s should say "N osds
> have noin set", and 'ceph health detail' should tell you which ones.
>
> Maybe something like
>
>  ceph osd set-osd osd.123 noin
>
> ?  I don't particularly like that but we can't do 'ceph osd set ...' since
> that does global osdmap flags.

I think we should make this operate on arbitrary named CRUSH nodes
rather than just OSDs, so that someone can mark a whole host/rack.

John

  reply	other threads:[~2016-01-20 13:55 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-01-20  5:42 Nodown/Noout by OSD_ID? Xiaoxi Chen
2016-01-20 13:32 ` Sage Weil
2016-01-20 13:55   ` John Spray [this message]
2016-01-20 15:26     ` Sage Weil
2016-01-21  1:47       ` Xiaoxi Chen
2016-01-21  1:58         ` Sage Weil

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALe9h7cqj+ck9-tKBgKuSx=4KcPSJnjO3O=eWSfV3yUT3J=QcQ@mail.gmail.com' \
    --to=jspray@redhat.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=sage@newdream.net \
    --cc=superdebuger@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.