SSDs and mdraid

SSDs and mdraid

[Finlayson, James M CIV (USA)]

All,
As I'm trying to achieve maximum performance on mdraid with SSDs, I've noticed a situation that I think could be corrected somewhat easily.

I've been having to play the partitioning game to get enough kernel workers to achieve maximum performance on mdraid SSD stripes, but I've run into a few troubling problems.   Basically on raid creation and on raid check, many events get DELAYED because they share underlying devices with other mdraid stripes when you look at the status in /proc/mdstat.   I feel like mdraid hasn't made the leap to SSDs, in that we have a signal in /sys/block/<md_device>/queue/rotational that  could enable  these DELAYED activities for SSDs.  The SSDs have way more IOPS, both read and write, to handle these DELAYs and we need to start taking advantage of the abilities of the SSDs.   It is an SSD world now.

Regards,
Jim


Jim Finlayson
U.S. Department of Defense

Re: SSDs and mdraid

[Phillip Susi]

"Finlayson, James M CIV (USA)" <james.m.finlayson4.civ@mail.mil> writes:

> All,
> As I'm trying to achieve maximum performance on mdraid with SSDs, I've
> noticed a situation that I think could be corrected somewhat easily.
>
> I've been having to play the partitioning game to get enough kernel
> workers to achieve maximum performance on mdraid SSD stripes, but I've
> run into a few troubling problems.  Basically on raid creation and on
> raid check, many events get DELAYED because they share underlying
> devices with other mdraid stripes when you look at the status in
> /proc/mdstat.  I feel like mdraid hasn't made the leap to SSDs, in
> that we have a signal in /sys/block/<md_device>/queue/rotational that
> could enable these DELAYED activities for SSDs.  The SSDs have way
> more IOPS, both read and write, to handle these DELAYs and we need to
> start taking advantage of the abilities of the SSDs.  It is an SSD
> world now.

While there is little to no pentalty for running multiple concurrent IO
streams to the SSD, there is nothing gained by doing so either.  In
other words, if you are trying to resync both mirrors on different parts
of two SSDs at the same time, each one will go half as fast, and will
take the same amount of time to finish both.

RE: [Non-DoD Source] Re: SSDs and mdraid

[Finlayson, James M CIV (USA)]

IF mdraid RESYNC ran as fast as the SSDs, but it doesn't :)

-----Original Message-----
From: Phillip Susi <phill@thesusis.net> 
Sent: Tuesday, August 24, 2021 3:39 PM
To: Finlayson, James M CIV (USA) <james.m.finlayson4.civ@mail.mil>
Cc: linux-raid@vger.kernel.org
Subject: [Non-DoD Source] Re: SSDs and mdraid


"Finlayson, James M CIV (USA)" <james.m.finlayson4.civ@mail.mil> writes:

> All,
> As I'm trying to achieve maximum performance on mdraid with SSDs, I've 
> noticed a situation that I think could be corrected somewhat easily.
>
> I've been having to play the partitioning game to get enough kernel 
> workers to achieve maximum performance on mdraid SSD stripes, but I've 
> run into a few troubling problems.  Basically on raid creation and on 
> raid check, many events get DELAYED because they share underlying 
> devices with other mdraid stripes when you look at the status in 
> /proc/mdstat.  I feel like mdraid hasn't made the leap to SSDs, in 
> that we have a signal in /sys/block/<md_device>/queue/rotational that 
> could enable these DELAYED activities for SSDs.  The SSDs have way 
> more IOPS, both read and write, to handle these DELAYs and we need to 
> start taking advantage of the abilities of the SSDs.  It is an SSD 
> world now.

While there is little to no pentalty for running multiple concurrent IO streams to the SSD, there is nothing gained by doing so either.  In other words, if you are trying to resync both mirrors on different parts of two SSDs at the same time, each one will go half as fast, and will take the same amount of time to finish both.