All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Blundell <David.Blundell@100percentit.com>
To: "ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Slow request warnings on 0.48
Date: Wed, 4 Jul 2012 16:53:02 +0000	[thread overview]
Message-ID: <45F11B42CB5830479EF0E56A27F4E68D7F2E2E@exchange1.ad.100it.net> (raw)

I have three servers running mon and osd using Ubuntu 12.04 that I have been testing with RADOS storing RBD KVM instances

0.47.3 worked extremely well (once I got over a few btrfs issues).  The same servers running 0.48 give a large number of "[WRN] slow request" messages whenever I generate a lot of random IO in the KVM instances using iozone.  The slow responses eventually leads to disk timeouts on the KVM instances.

I have erased the osds and recreated on new btrfs volumes with the same result.

I have also tried switching to xfs using mkfs.xfs -n size=64k with noatime, inode64,delaylog,logbufs=8,logbsize=256k

Xfs gives the same result - the iozone tests run fine until the random IO starts and then there are lots of slow request warnings.

Does anyone have any ideas about the best place to start troubleshooting / debugging?

Thanks,

David

             reply	other threads:[~2012-07-04 16:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-07-04 16:53 David Blundell [this message]
2012-07-04 16:58 ` Slow request warnings on 0.48 Alexandre DERUMIER
2012-07-04 18:59   ` Gregory Farnum
2012-07-04 22:03     ` David Blundell
2012-07-05 17:58   ` Mark Nelson
2012-07-05 18:16     ` David Blundell
2012-07-05 18:33     ` Alexandre DERUMIER
2012-07-05 18:43     ` David Blundell
2012-07-05 19:49       ` Mark Nelson
2012-07-05 20:21         ` Samuel Just
2012-07-05 22:15           ` David Blundell
2012-07-19 10:48 Matthew Richardson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=45F11B42CB5830479EF0E56A27F4E68D7F2E2E@exchange1.ad.100it.net \
    --to=david.blundell@100percentit.com \
    --cc=ceph-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.