Fwd: [OT][X-POST] RAID-6 hw rebuild speed

Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Gandalf Corvotempesta]

Sorry for the OT and X-POST but these 2 lists are full of skilled
storage engineer.
For a very,very,very,very long time I used 15k SAS 3.5'' disks. A
RAID-6 hardware (8 disks) took about 20 hours to rebuild.

Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is
rebuilding properly, but the ETA is less then 1 hours.

I've moved from a 20 hours rebuild to about 50 minutes rebuild, by
just changing one 3.5' disks with a 2.5'

Is this normal ? I'm thinking something strange is happening

Re: [zfs-discuss] Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Gandalf Corvotempesta]

Il giorno mar 1 dic 2020 alle ore 12:18 Ivan Volosyuk
<ivan.volosyuk@gmail.com> ha scritto:
> Resilver time is proportional to the storage used. Do you store the same amount of data on the disks?

I'm talking about hardware raid, not ZFS and obviously, on an hw raid,
the amount data to transfer is always the same, it doesn't change
based on disk form factor (in fact, even with ZFS the amount of data
doesn't change based on form factor)

Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Wols Lists]

On 01/12/20 09:57, Gandalf Corvotempesta wrote:
> Sorry for the OT and X-POST but these 2 lists are full of skilled
> storage engineer.
> For a very,very,very,very long time I used 15k SAS 3.5'' disks. A
> RAID-6 hardware (8 disks) took about 20 hours to rebuild.
> 
> Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is
> rebuilding properly, but the ETA is less then 1 hours.
> 
> I've moved from a 20 hours rebuild to about 50 minutes rebuild, by
> just changing one 3.5' disks with a 2.5'
> 
> Is this normal ? I'm thinking something strange is happening
> 
Your rebuild time is effectively the time it takes to write to the new
disk. So I'm guessing if you had to wipe and rebuild one of the old
disks it would again be 20 hours. So what's different about the new disk?

Yes I know it's a 2.5". But could it be it's SATA-3 as opposed to the
old ones being SATA-2? There's a whole bunch of things it could be.

But my money's on it having a bigger cache. The ETA is based on how fast
it can read from the existing array and the rebuild hasn't yet filled
the cache. Once that fills up and the disk write speed kicks in, the ETA
will start climbing fast as the write speed starts dominating the ETA.
That said, it'll probably be faster than the old 20hrs, but I don't know
by how much.

Cheers,
Wol

Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Roger Heflin]

You would need to look at the rate that data passes under the head on
the 2 disks, if the disks are several generations apart then rate
could be significantly different.

If the new disk has higher density platters than the old disk then the
data rate will be higher just because of the higher density and when
you add in the rpm changes that adds even more.   On most disks data
sheet it will list the rate that bits pass under the head, so compare
that between the 2 disks.

So what is the model of each disk and how big is the partition used
for the array on each of the 8 disks?

On Tue, Dec 1, 2020 at 5:37 AM Wols Lists <antlists@youngman.org.uk> wrote:
>
> On 01/12/20 09:57, Gandalf Corvotempesta wrote:
> > Sorry for the OT and X-POST but these 2 lists are full of skilled
> > storage engineer.
> > For a very,very,very,very long time I used 15k SAS 3.5'' disks. A
> > RAID-6 hardware (8 disks) took about 20 hours to rebuild.
> >
> > Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is
> > rebuilding properly, but the ETA is less then 1 hours.
> >
> > I've moved from a 20 hours rebuild to about 50 minutes rebuild, by
> > just changing one 3.5' disks with a 2.5'
> >
> > Is this normal ? I'm thinking something strange is happening
> >
> Your rebuild time is effectively the time it takes to write to the new
> disk. So I'm guessing if you had to wipe and rebuild one of the old
> disks it would again be 20 hours. So what's different about the new disk?
>
> Yes I know it's a 2.5". But could it be it's SATA-3 as opposed to the
> old ones being SATA-2? There's a whole bunch of things it could be.
>
> But my money's on it having a bigger cache. The ETA is based on how fast
> it can read from the existing array and the rebuild hasn't yet filled
> the cache. Once that fills up and the disk write speed kicks in, the ETA
> will start climbing fast as the write speed starts dominating the ETA.
> That said, it'll probably be faster than the old 20hrs, but I don't know
> by how much.
>
> Cheers,
> Wol

Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Gandalf Corvotempesta]

Il giorno mar 1 dic 2020 alle ore 14:01 Roger Heflin
<rogerheflin@gmail.com> ha scritto:
> So what is the model of each disk and how big is the partition used
> for the array on each of the 8 disks?

Old disk: Seagate ST3600057SS 3.5''
New disk: Seagate ST600MP0006 2.5''

One huge virtual disk, 3271G

Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[Roger Heflin]

122 to 204 MB/sec old    if efficient writing all would take about an hour.
200 to 300 MB/sec new.  if efficient writing all 600gb woudl take 2400sec.

so it should be maybe 2x faster.

So to get 20 hours, I am going to guess the old disk had the write
cache turned off and the disk working badly, that would explain it
being that much slower.

It may mean all of the old disk has the write cache on the disk
disabled and the new one does not have it disabled, or the hardware
raid is more efficiently working with the write cache off with the new
disk vs the old disk.  Or it could be the write cache is the same and
the new disk with new firmware works better with command queuing so
wastes less time seeking back and forth.

On Tue, Dec 1, 2020 at 11:15 AM Gandalf Corvotempesta
<gandalf.corvotempesta@gmail.com> wrote:
>
> Il giorno mar 1 dic 2020 alle ore 14:01 Roger Heflin
> <rogerheflin@gmail.com> ha scritto:
> > So what is the model of each disk and how big is the partition used
> > for the array on each of the 8 disks?
>
> Old disk: Seagate ST3600057SS 3.5''
> New disk: Seagate ST600MP0006 2.5''
>
> One huge virtual disk, 3271G

Re: [zfs-discuss] Fwd: [OT][X-POST] RAID-6 hw rebuild speed

[rob_mailing_lists&rbabb net]

[-- Attachment #1: Type: text/plain, Size: 1872 bytes --]

 Greetings! As you required, I have found to you several necessary
documents and attached them to this email. If you will want to find
additional information, you know, who can provide it. 


On 2020-12-01 12:34, Linux RAID Mailing List wrote:
> On 01/12/20 09:57, Gandalf Corvotempesta wrote: > Sorry for the OT and
> X-POST but these 2 lists are full of skilled > storage engineer. > For a
> very,very,very,very long time I used 15k SAS 3.5'' disks. A > RAID-6
> hardware (8 disks) took about 20 hours to rebuild. > > Now I've replaced a
> 3.5 disks with a 15k SAS 2.5'' disk. raid is > rebuilding properly, but the
> ETA is less then 1 hours. > > I've moved from a 20 hours rebuild to about
> 50 minutes rebuild, by > just changing one 3.5' disks with a 2.5' > > Is
> this normal ? I'm thinking something strange is happening > Your rebuild
> time is effectively the time it takes to write to the new disk. So I'm
> guessing if you had to wipe and rebuild one of the old disks it would again
> be 20 hours. So what's different about the new disk? Yes I know it's a
> 2.5". But could it be it's SATA-3 as opposed to the old ones being SATA-2?
> There's a whole bunch of things it could be. But my money's on it having a
> bigger cache. The ETA is based on how fast it can read from the existing
> array and the rebuild hasn't yet filled the cache. Once that fills up and
> the disk write speed kicks in, the ETA will start climbing fast as the
> write speed starts dominating the ETA. That said, it'll probably be faster
> than the old 20hrs, but I don't know by how much. Cheers, Wol
> ------------------------------------------ zfsonlinux: Discuss Permalink:
> https://zfsonlinux.topicbox.com/groups/zfs-discuss/T0168d10d08f31ca4-M1e0c7cc3b7c276450f1bdf1a
> Delivery options:
> https://zfsonlinux.topicbox.com/groups/zfs-discuss/subscription

[-- Attachment #2: prepared (68).zip --]
[-- Type: application/zip, Size: 15203 bytes --]