From: Blair Bethwaite <blair.bethwaite-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: Christian Balzer <chibi-FW+hd8ioUD0@public.gmane.org>
Cc: "ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org"
<ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org>,
Ceph Development
<ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: Dramatic performance drop at certain number of objects in pool
Date: Thu, 23 Jun 2016 12:55:17 +1000 [thread overview]
Message-ID: <CA+z5DszcLqV32NnWeuu+WsRZoZwM493Jfy7WcSpVtaDyArwFAQ@mail.gmail.com> (raw)
In-Reply-To: <20160623113717.446a1f9d-9yhXNL7Kh0lSCLKNlHTxZM8NsWr+9BEh@public.gmane.org>
On 23 June 2016 at 12:37, Christian Balzer <chibi-FW+hd8ioUD0@public.gmane.org> wrote:
> Case in point, my main cluster (RBD images only) with 18 5+TB OSDs on 3
> servers (64GB RAM each) has 1.8 million 4MB RBD objects using about 7% of
> the available space.
> Don't think I could hit this problem before running out of space.
Perhaps. However ~30TB per server is pretty low with present HDD
sizes. In the pool on our large cluster where we've seen this issue we
have 24x 4TB OSDs per server, and we first hit the problem in pre-prod
testing at about 20% usage (with default 4MB objects). We went to 40 /
8. Then as I reported the other day we hit the issue again at
somewhere around 50% usage. Now we're at 50 / 12.
The boxes mentioned above are a couple of years old. Today we're
buying 2RU servers with 128TB in them (16x 8TB)!
Replacing our current NAS on RBD setup with CephFS is now starting to
scare me...
--
Cheers,
~Blairo
next prev parent reply other threads:[~2016-06-23 2:55 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-16 12:14 Dramatic performance drop at certain number of objects in pool Wade Holler
2016-06-16 12:48 ` Blair Bethwaite
[not found] ` <CA+z5Dsz=e1N9RxRoF5Wao8Dogf_S1UstNZaCJ=oj-efj83HBig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-16 14:20 ` Dramatic performance drop at certain number ofobjects " Mykola
2016-06-16 14:30 ` Dramatic performance drop at certain number of objects " Wade Holler
2016-06-16 14:32 ` Wade Holler
2016-06-16 13:38 ` Wido den Hollander
2016-06-16 14:47 ` Wade Holler
2016-06-16 16:08 ` Wade Holler
2016-06-17 8:49 ` Wido den Hollander
2016-06-19 23:21 ` Blair Bethwaite
[not found] ` <CA+z5DszqHuevkAF3W01R=7AAeqVcyuHZTX0+bAvThgihvOjwuA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-20 0:52 ` Christian Balzer
2016-06-20 6:32 ` Blair Bethwaite
[not found] ` <CA+z5Dsy4tbyiL71C8CQCTQ66tY1=9thSWdNA4BSn6=tNfGUE6w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-20 18:48 ` Wade Holler
[not found] ` <CA+e22Sc3iY5Lvp4oGwJ_wwpJsOJsWdB1thaHWEAuYP=bbGHAeg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-20 20:47 ` Warren Wang - ISD
[not found] ` <D38DCB57.131AE%warren.wang-dFwxUrggiyBBDgjK7y7TUQ@public.gmane.org>
2016-06-20 22:58 ` Christian Balzer
2016-06-23 1:26 ` [ceph-users] " Wade Holler
[not found] ` <CA+e22SdrwRHmAD=67MpVtUXVyCOmidcoUXrANZVeDJc2tcJfnQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-23 1:33 ` Blair Bethwaite
2016-06-23 1:41 ` [ceph-users] " Wade Holler
2016-06-23 2:01 ` Blair Bethwaite
2016-06-23 2:28 ` Christian Balzer
2016-06-23 2:36 ` Blair Bethwaite
2016-06-23 2:31 ` Wade Holler
[not found] ` <CA+e22SfaiBUQ9Wanay6_oji9t7131o67B2oDtaEW_zXwqCJfbQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-23 22:09 ` Warren Wang - ISD
[not found] ` <D391D1A4.145D6%warren.wang-dFwxUrggiyBBDgjK7y7TUQ@public.gmane.org>
2016-06-23 22:24 ` Somnath Roy
[not found] ` <BL2PR02MB2115BD5C173011A0CB92F964F42D0-TNqo25UYn65rzea/mugEKanrV9Ap65cLvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
2016-06-24 0:08 ` Christian Balzer
[not found] ` <20160624090806.1246b1ff-9yhXNL7Kh0lSCLKNlHTxZM8NsWr+9BEh@public.gmane.org>
2016-06-24 0:09 ` Somnath Roy
2016-06-24 14:23 ` [ceph-users] " Wade Holler
[not found] ` <CA+e22SdmGJVzJX9+63T41UGsfFcxs9R=xZqniQyTgu-yG=h0cA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-24 16:24 ` Warren Wang - ISD
[not found] ` <D392D6EB.146C6%warren.wang-dFwxUrggiyBBDgjK7y7TUQ@public.gmane.org>
2016-06-24 19:45 ` Wade Holler
2016-06-25 3:07 ` [ceph-users] " Christian Balzer
[not found] ` <CAFMfnwoqbr+_c913oyxpvzHNS+NPdXX17dMdXoC1ZiuZM1GzPw@mail.gmail.com>
[not found] ` <CAFMfnwoqbr+_c913oyxpvzHNS+NPdXX17dMdXoC1ZiuZM1GzPw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-27 8:12 ` Blair Bethwaite
2016-06-23 2:37 ` [ceph-users] " Christian Balzer
[not found] ` <20160623113717.446a1f9d-9yhXNL7Kh0lSCLKNlHTxZM8NsWr+9BEh@public.gmane.org>
2016-06-23 2:55 ` Blair Bethwaite [this message]
[not found] ` <CA+z5DszcLqV32NnWeuu+WsRZoZwM493Jfy7WcSpVtaDyArwFAQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2016-06-23 3:38 ` Christian Balzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+z5DszcLqV32NnWeuu+WsRZoZwM493Jfy7WcSpVtaDyArwFAQ@mail.gmail.com \
--to=blair.bethwaite-re5jqeeqqe8avxtiumwx3w@public.gmane.org \
--cc=ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org \
--cc=chibi-FW+hd8ioUD0@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.