linux-lvm.redhat.com archive mirror
 help / color / mirror / Atom feed
From: Anatoly Pugachev <matorola@gmail.com>
To: LVM general discussion and development <linux-lvm@redhat.com>
Subject: Re: [linux-lvm] Best way to run LVM over multiple SW RAIDs?
Date: Sat, 7 Dec 2019 19:16:20 +0300	[thread overview]
Message-ID: <CADxRZqyWbvoGGVJXdx9Mj+9toTM8tSMP9oUE8F0b+OhYqJ4uUA@mail.gmail.com> (raw)
In-Reply-To: <A8A53B0E4B7ACF4BAD5C77D3671AC8976AB85B3B@EX01.edgeware.tv>

On Tue, Oct 29, 2019 at 12:14 PM Daniel Janzon
<daniel.janzon@edgeware.tv> wrote:
>
> Hello,
>
> I have a server with very high load using four NVMe SSDs and therefore no HW RAID. Instead I used SW RAID with the mdadm tool. Using one RAID5 volume does not work well since the driver can only utilize one CPU core which spikes at 100% and harms performance. Therefore I created 8 partitions on each disk, and 8 RAID5s across the four disks.
>
> Now I want to bring them together with LVM. If I do not use a striped volume I get high performance (in expected magnitude according to disk specs). But when I use a striped volume, performance drops to a magnitude below. The reason I am looking for a striped setup is to ensure that data is spread well over the drive to guarantee a good worst-case performance. With linear allocation rather than striped, if load is directed to files on the first PV (a SW RAID) the system is again exposed to the 1-core limitation.
>
> I tried "--stripes 8 --stripesize 512", and would appreciate any ideas of other things to try. I guess the performance hit can be in the file system as well. I tried XFS and EXT4 with default settings.

Daniel,

a bit more about your system? Like kernel version, io scheduler, etc..
Have you tried with recent kernels MQ (multi-queue) schedulers (noop,
deadline) ?

  reply	other threads:[~2019-12-07 16:16 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-29  8:47 [linux-lvm] Best way to run LVM over multiple SW RAIDs? Daniel Janzon
2019-12-07 16:16 ` Anatoly Pugachev [this message]
2019-12-07 17:37   ` Roberto Fastec
2019-12-07 20:34     ` Stuart D. Gathman
2019-12-07 22:44       ` John Stoffel
2019-12-07 23:14         ` Stuart D. Gathman
2019-12-08 11:57           ` Gionatan Danti
2019-12-08 22:51           ` John Stoffel
2019-12-09 10:40         ` Guoqing Jiang
2019-12-09 10:26 Daniel Janzon
2019-12-09 14:26 ` Marian Csontos
2019-12-10 11:23 ` Gionatan Danti
2019-12-10 21:29   ` John Stoffel
2019-12-16  8:22 Daniel Janzon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CADxRZqyWbvoGGVJXdx9Mj+9toTM8tSMP9oUE8F0b+OhYqJ4uUA@mail.gmail.com \
    --to=matorola@gmail.com \
    --cc=linux-lvm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).