All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stefan Priebe <s.priebe@profihost.ag>
To: Sage Weil <sage@inktank.com>
Cc: "ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: ceph-osd cpu usage
Date: Thu, 15 Nov 2012 21:26:06 +0100	[thread overview]
Message-ID: <50A54FDE.9070403@profihost.ag> (raw)
In-Reply-To: <alpine.DEB.2.00.1211150710550.4011@cobra.newdream.net>

Am 15.11.2012 16:14, schrieb Sage Weil:
> On Thu, 15 Nov 2012, Stefan Priebe - Profihost AG wrote:
> Hmm, most significant time seems to be in the allocator and doing
> fsetxattr(2) (10%!).  Also some path traversal stuff.
Yes fsetxattr seems to be CPU hungry.

> Can you try the wip-fd-simple-cache branch, which tries to spend less time
> closing and reopening files?  I'm curious how much of a different it will
> make for you for both IOPS and CPU utilization.
It seems to give me around 1000 iops across 3 VMs.

> It is also possible to use leveldb for most attrs.  If you set
> 'filestore xattr use omap = true' it should put most attrs in leveldb.
Tried this but this raises CPU by 20%.

Any other ideas how to reduce ceph-osd while doing randwrite?

Randread gives me with 3 VMs: 60.000 iops
Randwrite gives me with 3 VMs: 25.000 iops

Stefan

  parent reply	other threads:[~2012-11-15 20:26 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-11-15 10:56 ceph-osd cpu usage Stefan Priebe - Profihost AG
2012-11-15 11:18 ` Alexandre DERUMIER
2012-11-15 12:19   ` Stefan Priebe - Profihost AG
2012-11-15 15:12     ` Mark Nelson
2012-11-15 16:09       ` Stefan Priebe
2012-11-15 16:52         ` Sage Weil
2012-11-15 19:44       ` Stefan Priebe
2012-11-15 15:14 ` Sage Weil
2012-11-15 15:30   ` Stefan Priebe - Profihost AG
2012-11-15 20:26   ` Stefan Priebe [this message]
2012-11-16  8:41     ` Alexandre DERUMIER
2012-11-16  9:11       ` Stefan Priebe - Profihost AG

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=50A54FDE.9070403@profihost.ag \
    --to=s.priebe@profihost.ag \
    --cc=ceph-devel@vger.kernel.org \
    --cc=sage@inktank.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.