From: Yann Dupont <Yann.Dupont@univ-nantes.fr>
To: Peter Grandi <pg_xf2@xf2.for.sabi.co.UK>
Cc: Linux fs XFS <xfs@oss.sgi.com>
Subject: Re: Bad performance with XFS + 2.6.38 / 2.6.39
Date: Wed, 04 Jan 2012 11:54:43 +0100 [thread overview]
Message-ID: <4F042FF3.7090104@univ-nantes.fr> (raw)
In-Reply-To: <20225.54924.482210.587313@tree.ty.sabi.co.UK>
On 02/01/2012 17:08, Peter Grandi wrote:
> [ ... ]
>
>>> On two particular server, with recent kernels, I experience a
>>> much higher load than expected, but it's very hard to tell
>>> what's wrong. The system seems more in I/O wait. Older
>>> kernels (2.6.32.xx and 2.6.26.xx) gives better results.
> [ ... ]
>> When I go back to older kernels, the load go down. With newer
>> kernel, all is working well too, but load (as reported by
>> uptime) is higher.
> [ ... ]
>>> birnie:~/TRACE# uptime
>>> 11:48:34 up 17:18, 3 users, load average: 0.04, 0.18, 0.23
>
>>> penderyn:~/TRACE# uptime
>>> 11:48:30 up 23 min, 3 users, load average: 4.03, 3.82, 3.21
> [ ... ]
>
> But 'uptime' reports the load average, which is (roughly)
> processes actually running on the CPU. If the load average is
More or less. I generally have 5000+ processes on those servers. The
load generally reflect a mix between CPU usage (which is unchanged as
dovecot setup is unchanged) and I/O wait. So naively, I'll say if load
average is higher than usual, that's because I/O WAIT is higher.
As kernel had big changes, it could be XFS, but DM, or I/O scheduler as
well.
But it don't seems the case.
> higher, that usually means that the file system is running
> better, not worse.
If delivery is I/O bound, yes but that's not the case in this particular
setup.
It looks as if you are not clear whether you
> have a regression or an improvement.
I was just signaling an unusual load average, nothing else. As far as I
can see, response times are still correct. I'm not experiencing a
performance proble. I'm not the first author of the thread. I probably
should have changed the name of the thread, sorry for that.
>
> For a mail server the relevant metric is messages processed per
> second, or alternatively median and maximum times to process a
> message, rather than "average" processes running.
>
...
> So you are expecting for a large system critical problem for
> which you yourself do not have the resource to do testing to see
> quick response times over the Christmas and New Year period.
> What's your XFS Platinum Psychic Support Account number? :-)
I'm not expecting anything. I know open source. All is working fine,
thank you. I was just "upping" because I saw that my traces have been
downloaded last week. It's not always easy for non native speakers to
send mails without sounding agressive/offendant . If that was the case,I
can assure that was not the intent.
>
> BTW rereading the description of the setup:
>
>>>>>> Thoses servers are mail (dovecot) servers, with lots of
>>>>>> simultaneous imap clients (5000+) an lots of simultaneous
>>>>>> message delivery. These are linux-vservers, on top of LVM
>>>>>> volumes. The storage is SAN with 15k RPM SAS drives (and
>>>>>> battery backup). I know barriers were disabled in older
>>>>>> kernels, so with recents kernels, XFS volumes were mounted
>>>>>> with nobarrier.
>
>>>>> 1. What mailbox format are you using? Is this a constant
>>>>> or variable?
>>>> Maildir++
>
> I am stunned by the sheer (euphemism alert) audacity of it all.
> This setup is (euphemism alert) amazing.
Can you elaborate, please ?? This particular setup is running fine for 7
years now , has very finely scaled up (up to 70k mailboxes with a
similar setup for students) with little modifications (replacing
courrier by dovecot, and upgrading servers for example) and has proved
very stable since, despite numerous power outages, for example...
I can give you detailed setup if you want, off list, I think it has
nothing to do with xfs.
>
> Unfortunately the problem of large busy mailstores is vastly
> underestimated by many, and XFS has little to do with it.
>
really not sure I underestimate it, but I'll glad to hear your
recommendations. Offlist, I think.
Cheers,
--
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@univ-nantes.fr
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2012-01-04 10:57 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-11 12:45 Bad performance with XFS + 2.6.38 / 2.6.39 Xupeng Yun
2011-12-11 23:39 ` Dave Chinner
2011-12-12 0:40 ` Xupeng Yun
2011-12-12 1:00 ` Dave Chinner
2011-12-12 2:00 ` Xupeng Yun
2011-12-12 13:57 ` Christoph Hellwig
2011-12-21 9:08 ` Yann Dupont
2011-12-21 15:10 ` Stan Hoeppner
2011-12-21 17:56 ` Yann Dupont
2011-12-21 22:26 ` Dave Chinner
2011-12-22 9:23 ` Yann Dupont
2011-12-22 11:02 ` Yann Dupont
2012-01-02 10:06 ` Yann Dupont
2012-01-02 16:08 ` Peter Grandi
2012-01-02 18:02 ` Peter Grandi
2012-01-04 10:54 ` Yann Dupont [this message]
2012-01-02 20:35 ` Dave Chinner
2012-01-03 8:20 ` Yann Dupont
2012-01-04 12:33 ` Christoph Hellwig
2012-01-04 13:06 ` Yann Dupont
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4F042FF3.7090104@univ-nantes.fr \
--to=yann.dupont@univ-nantes.fr \
--cc=pg_xf2@xf2.for.sabi.co.UK \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.