* Fwd: [OT][X-POST] RAID-6 hw rebuild speed [not found] <CAJH6TXjsg+OE5rUpK+RqeFJRxBiZJ94ToOdUD5ajjwXzYft9Vw@mail.gmail.com> @ 2020-12-01 9:57 ` Gandalf Corvotempesta [not found] ` <CAJ1=CigDVO9-2uSBw8Fbv-86y8G6XOFM2CjRs1yURAczgB6ydA@mail.gmail.com> 2020-12-01 11:34 ` Wols Lists 0 siblings, 2 replies; 7+ messages in thread From: Gandalf Corvotempesta @ 2020-12-01 9:57 UTC (permalink / raw) To: General discussion - ask questions, receive answers and advice from other ZFS users, Linux RAID Mailing List Sorry for the OT and X-POST but these 2 lists are full of skilled storage engineer. For a very,very,very,very long time I used 15k SAS 3.5'' disks. A RAID-6 hardware (8 disks) took about 20 hours to rebuild. Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is rebuilding properly, but the ETA is less then 1 hours. I've moved from a 20 hours rebuild to about 50 minutes rebuild, by just changing one 3.5' disks with a 2.5' Is this normal ? I'm thinking something strange is happening ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <CAJ1=CigDVO9-2uSBw8Fbv-86y8G6XOFM2CjRs1yURAczgB6ydA@mail.gmail.com>]
* Re: [zfs-discuss] Fwd: [OT][X-POST] RAID-6 hw rebuild speed [not found] ` <CAJ1=CigDVO9-2uSBw8Fbv-86y8G6XOFM2CjRs1yURAczgB6ydA@mail.gmail.com> @ 2020-12-01 11:28 ` Gandalf Corvotempesta 0 siblings, 0 replies; 7+ messages in thread From: Gandalf Corvotempesta @ 2020-12-01 11:28 UTC (permalink / raw) To: Discuss; +Cc: Linux RAID Mailing List Il giorno mar 1 dic 2020 alle ore 12:18 Ivan Volosyuk <ivan.volosyuk@gmail.com> ha scritto: > Resilver time is proportional to the storage used. Do you store the same amount of data on the disks? I'm talking about hardware raid, not ZFS and obviously, on an hw raid, the amount data to transfer is always the same, it doesn't change based on disk form factor (in fact, even with ZFS the amount of data doesn't change based on form factor) ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed 2020-12-01 9:57 ` Fwd: [OT][X-POST] RAID-6 hw rebuild speed Gandalf Corvotempesta [not found] ` <CAJ1=CigDVO9-2uSBw8Fbv-86y8G6XOFM2CjRs1yURAczgB6ydA@mail.gmail.com> @ 2020-12-01 11:34 ` Wols Lists 2020-12-01 13:01 ` Roger Heflin 2021-02-26 14:04 ` [zfs-discuss] " rob_mailing_lists&rbabb net 1 sibling, 2 replies; 7+ messages in thread From: Wols Lists @ 2020-12-01 11:34 UTC (permalink / raw) To: Gandalf Corvotempesta, General discussion - ask questions, receive answers and advice from other ZFS users, Linux RAID Mailing List On 01/12/20 09:57, Gandalf Corvotempesta wrote: > Sorry for the OT and X-POST but these 2 lists are full of skilled > storage engineer. > For a very,very,very,very long time I used 15k SAS 3.5'' disks. A > RAID-6 hardware (8 disks) took about 20 hours to rebuild. > > Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is > rebuilding properly, but the ETA is less then 1 hours. > > I've moved from a 20 hours rebuild to about 50 minutes rebuild, by > just changing one 3.5' disks with a 2.5' > > Is this normal ? I'm thinking something strange is happening > Your rebuild time is effectively the time it takes to write to the new disk. So I'm guessing if you had to wipe and rebuild one of the old disks it would again be 20 hours. So what's different about the new disk? Yes I know it's a 2.5". But could it be it's SATA-3 as opposed to the old ones being SATA-2? There's a whole bunch of things it could be. But my money's on it having a bigger cache. The ETA is based on how fast it can read from the existing array and the rebuild hasn't yet filled the cache. Once that fills up and the disk write speed kicks in, the ETA will start climbing fast as the write speed starts dominating the ETA. That said, it'll probably be faster than the old 20hrs, but I don't know by how much. Cheers, Wol ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed 2020-12-01 11:34 ` Wols Lists @ 2020-12-01 13:01 ` Roger Heflin 2020-12-01 17:15 ` Gandalf Corvotempesta 2021-02-26 14:04 ` [zfs-discuss] " rob_mailing_lists&rbabb net 1 sibling, 1 reply; 7+ messages in thread From: Roger Heflin @ 2020-12-01 13:01 UTC (permalink / raw) To: Wols Lists Cc: Gandalf Corvotempesta, General discussion - ask questions, receive answers and advice from other ZFS users, Linux RAID Mailing List You would need to look at the rate that data passes under the head on the 2 disks, if the disks are several generations apart then rate could be significantly different. If the new disk has higher density platters than the old disk then the data rate will be higher just because of the higher density and when you add in the rpm changes that adds even more. On most disks data sheet it will list the rate that bits pass under the head, so compare that between the 2 disks. So what is the model of each disk and how big is the partition used for the array on each of the 8 disks? On Tue, Dec 1, 2020 at 5:37 AM Wols Lists <antlists@youngman.org.uk> wrote: > > On 01/12/20 09:57, Gandalf Corvotempesta wrote: > > Sorry for the OT and X-POST but these 2 lists are full of skilled > > storage engineer. > > For a very,very,very,very long time I used 15k SAS 3.5'' disks. A > > RAID-6 hardware (8 disks) took about 20 hours to rebuild. > > > > Now I've replaced a 3.5 disks with a 15k SAS 2.5'' disk. raid is > > rebuilding properly, but the ETA is less then 1 hours. > > > > I've moved from a 20 hours rebuild to about 50 minutes rebuild, by > > just changing one 3.5' disks with a 2.5' > > > > Is this normal ? I'm thinking something strange is happening > > > Your rebuild time is effectively the time it takes to write to the new > disk. So I'm guessing if you had to wipe and rebuild one of the old > disks it would again be 20 hours. So what's different about the new disk? > > Yes I know it's a 2.5". But could it be it's SATA-3 as opposed to the > old ones being SATA-2? There's a whole bunch of things it could be. > > But my money's on it having a bigger cache. The ETA is based on how fast > it can read from the existing array and the rebuild hasn't yet filled > the cache. Once that fills up and the disk write speed kicks in, the ETA > will start climbing fast as the write speed starts dominating the ETA. > That said, it'll probably be faster than the old 20hrs, but I don't know > by how much. > > Cheers, > Wol ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed 2020-12-01 13:01 ` Roger Heflin @ 2020-12-01 17:15 ` Gandalf Corvotempesta 2020-12-02 1:21 ` Roger Heflin 0 siblings, 1 reply; 7+ messages in thread From: Gandalf Corvotempesta @ 2020-12-01 17:15 UTC (permalink / raw) To: Roger Heflin Cc: Wols Lists, General discussion - ask questions, receive answers and advice from other ZFS users, Linux RAID Mailing List Il giorno mar 1 dic 2020 alle ore 14:01 Roger Heflin <rogerheflin@gmail.com> ha scritto: > So what is the model of each disk and how big is the partition used > for the array on each of the 8 disks? Old disk: Seagate ST3600057SS 3.5'' New disk: Seagate ST600MP0006 2.5'' One huge virtual disk, 3271G ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Fwd: [OT][X-POST] RAID-6 hw rebuild speed 2020-12-01 17:15 ` Gandalf Corvotempesta @ 2020-12-02 1:21 ` Roger Heflin 0 siblings, 0 replies; 7+ messages in thread From: Roger Heflin @ 2020-12-02 1:21 UTC (permalink / raw) To: Gandalf Corvotempesta Cc: Wols Lists, General discussion - ask questions, receive answers and advice from other ZFS users, Linux RAID Mailing List 122 to 204 MB/sec old if efficient writing all would take about an hour. 200 to 300 MB/sec new. if efficient writing all 600gb woudl take 2400sec. so it should be maybe 2x faster. So to get 20 hours, I am going to guess the old disk had the write cache turned off and the disk working badly, that would explain it being that much slower. It may mean all of the old disk has the write cache on the disk disabled and the new one does not have it disabled, or the hardware raid is more efficiently working with the write cache off with the new disk vs the old disk. Or it could be the write cache is the same and the new disk with new firmware works better with command queuing so wastes less time seeking back and forth. On Tue, Dec 1, 2020 at 11:15 AM Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com> wrote: > > Il giorno mar 1 dic 2020 alle ore 14:01 Roger Heflin > <rogerheflin@gmail.com> ha scritto: > > So what is the model of each disk and how big is the partition used > > for the array on each of the 8 disks? > > Old disk: Seagate ST3600057SS 3.5'' > New disk: Seagate ST600MP0006 2.5'' > > One huge virtual disk, 3271G ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [zfs-discuss] Fwd: [OT][X-POST] RAID-6 hw rebuild speed 2020-12-01 11:34 ` Wols Lists 2020-12-01 13:01 ` Roger Heflin @ 2021-02-26 14:04 ` rob_mailing_lists&rbabb net 1 sibling, 0 replies; 7+ messages in thread From: rob_mailing_lists&rbabb net @ 2021-02-26 14:04 UTC (permalink / raw) To: Linux RAID Mailing List [-- Attachment #1: Type: text/plain, Size: 1872 bytes --] Greetings! As you required, I have found to you several necessary documents and attached them to this email. If you will want to find additional information, you know, who can provide it. On 2020-12-01 12:34, Linux RAID Mailing List wrote: > On 01/12/20 09:57, Gandalf Corvotempesta wrote: > Sorry for the OT and > X-POST but these 2 lists are full of skilled > storage engineer. > For a > very,very,very,very long time I used 15k SAS 3.5'' disks. A > RAID-6 > hardware (8 disks) took about 20 hours to rebuild. > > Now I've replaced a > 3.5 disks with a 15k SAS 2.5'' disk. raid is > rebuilding properly, but the > ETA is less then 1 hours. > > I've moved from a 20 hours rebuild to about > 50 minutes rebuild, by > just changing one 3.5' disks with a 2.5' > > Is > this normal ? I'm thinking something strange is happening > Your rebuild > time is effectively the time it takes to write to the new disk. So I'm > guessing if you had to wipe and rebuild one of the old disks it would again > be 20 hours. So what's different about the new disk? Yes I know it's a > 2.5". But could it be it's SATA-3 as opposed to the old ones being SATA-2? > There's a whole bunch of things it could be. But my money's on it having a > bigger cache. The ETA is based on how fast it can read from the existing > array and the rebuild hasn't yet filled the cache. Once that fills up and > the disk write speed kicks in, the ETA will start climbing fast as the > write speed starts dominating the ETA. That said, it'll probably be faster > than the old 20hrs, but I don't know by how much. Cheers, Wol > ------------------------------------------ zfsonlinux: Discuss Permalink: > https://zfsonlinux.topicbox.com/groups/zfs-discuss/T0168d10d08f31ca4-M1e0c7cc3b7c276450f1bdf1a > Delivery options: > https://zfsonlinux.topicbox.com/groups/zfs-discuss/subscription [-- Attachment #2: prepared (68).zip --] [-- Type: application/zip, Size: 15203 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-02-26 14:11 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAJH6TXjsg+OE5rUpK+RqeFJRxBiZJ94ToOdUD5ajjwXzYft9Vw@mail.gmail.com> 2020-12-01 9:57 ` Fwd: [OT][X-POST] RAID-6 hw rebuild speed Gandalf Corvotempesta [not found] ` <CAJ1=CigDVO9-2uSBw8Fbv-86y8G6XOFM2CjRs1yURAczgB6ydA@mail.gmail.com> 2020-12-01 11:28 ` [zfs-discuss] " Gandalf Corvotempesta 2020-12-01 11:34 ` Wols Lists 2020-12-01 13:01 ` Roger Heflin 2020-12-01 17:15 ` Gandalf Corvotempesta 2020-12-02 1:21 ` Roger Heflin 2021-02-26 14:04 ` [zfs-discuss] " rob_mailing_lists&rbabb net
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).