From: Christian Kauhaus <kc-4ImjS3E6Ur3YxmODkEiZ1Q@public.gmane.org>
To: Robert LeBlanc <robert-4JaGZRWAfWbajFs6igw21g@public.gmane.org>
Cc: Ceph-User <ceph-users-Qp0mS5GaXlQ@public.gmane.org>,
ceph-devel <ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: Blocked requests after "osd in"
Date: Thu, 10 Dec 2015 10:03:51 +0100 [thread overview]
Message-ID: <56693FF7.8050708@flyingcircus.io> (raw)
In-Reply-To: <CAANLjFqUoGTFSYrdwCwRw0apfXJN61SeS7FiXb0A_Bxtu4PJJg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 1341 bytes --]
Am 10.12.2015 um 06:38 schrieb Robert LeBlanc:
> I noticed this a while back and did some tracing. As soon as the PGs
> are read in by the OSD (very limited amount of housekeeping done), the
> OSD is set to the "in" state so that peering with other OSDs can
> happen and the recovery process can begin. The problem is that when
> the OSD is "in", the clients also see that and start sending requests
> to the OSDs before it has had a chance to actually get its bearings
> and is able to even service the requests. After discussion with some
> of the developers, there is no easy way around this other than let the
> PGs recover to other OSDs and then bring in the OSDs after recovery (a
> ton of data movement).
Many thanks for your detailed analysis. It's a bit disappointing that there
seems to be no easy way around. Any work to improve the situation is much
appreciated.
In the meantime, I'll be experimenting with pre-seeding the VFS cache to speed
things up at least a little bit.
Regards
Christian
--
Dipl-Inf. Christian Kauhaus <>< · kc-4ImjS3E6Ur3YxmODkEiZ1Q@public.gmane.org · +49 345 219401-0
Flying Circus Internet Operations GmbH · http://flyingcircus.io
Forsterstraße 29 · 06112 Halle (Saale) · Deutschland
HR Stendal 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
[-- Attachment #2: Type: text/plain, Size: 178 bytes --]
_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
next prev parent reply other threads:[~2015-12-10 9:03 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <5667FC6E.5080600@flyingcircus.io>
[not found] ` <B0751415-538E-471E-B310-76812C3C374C@schermer.cz>
[not found] ` <56683B9E.8070201@flyingcircus.io>
2015-12-10 5:38 ` [ceph-users] Blocked requests after "osd in" Robert LeBlanc
[not found] ` <CAANLjFqUoGTFSYrdwCwRw0apfXJN61SeS7FiXb0A_Bxtu4PJJg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2015-12-10 9:03 ` Christian Kauhaus [this message]
2015-12-10 9:58 ` Jan Schermer
2015-12-11 14:14 ` Christian Kauhaus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56693FF7.8050708@flyingcircus.io \
--to=kc-4imjs3e6ur3yxmodkeiz1q@public.gmane.org \
--cc=ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=ceph-users-Qp0mS5GaXlQ@public.gmane.org \
--cc=robert-4JaGZRWAfWbajFs6igw21g@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.