new drive is 4 sectors shorter, can it be used for swraid5 array?

new drive is 4 sectors shorter, can it be used for swraid5 array?

[Marc MERLIN]

old drive:
Device Model:     ST6000VN0041-2EL11C
Serial Number:    ZA18WX4T
LU WWN Device Id: 5 000c50 0a47d527a
Firmware Version: SC61
User Capacity:    6,001,175,126,016 bytes [6.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical

   8      128 5860522584 sdi
   8      129 5860521543 sdi1


new drive:
Device Model:     ST6000VN001-2BB186
Serial Number:    ZR118A1Y
LU WWN Device Id: 5 000c50 0dba1b3c0
Firmware Version: SC60
User Capacity:    6,001,175,126,016 bytes [6.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical

   8      160 5860522580 sdk
   8      161 5860521536 sdk1

New drive is 4 sectors shorter, so I assume I can't use it as a replacement in my md5
array because it's 4 sectors too short, or does swraid5 not need the last few sectors
of a drive?

Looks like formatting as MDR won't help, I'm still 4 sectors short.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Wols Lists]

On 18/03/2022 03:08, Marc MERLIN wrote:
> old drive:
> Device Model:     ST6000VN0041-2EL11C
> Serial Number:    ZA18WX4T
> LU WWN Device Id: 5 000c50 0a47d527a
> Firmware Version: SC61
> User Capacity:    6,001,175,126,016 bytes [6.00 TB]
> Sector Sizes:     512 bytes logical, 4096 bytes physical
> 
>     8      128 5860522584 sdi
>     8      129 5860521543 sdi1
> 
> 
> new drive:
> Device Model:     ST6000VN001-2BB186
> Serial Number:    ZR118A1Y
> LU WWN Device Id: 5 000c50 0dba1b3c0
> Firmware Version: SC60
> User Capacity:    6,001,175,126,016 bytes [6.00 TB]
> Sector Sizes:     512 bytes logical, 4096 bytes physical
> 
>     8      160 5860522580 sdk
>     8      161 5860521536 sdk1
> 
> New drive is 4 sectors shorter, so I assume I can't use it as a replacement in my md5
> array because it's 4 sectors too short, or does swraid5 not need the last few sectors
> of a drive?
> 
> Looks like formatting as MDR won't help, I'm still 4 sectors short.
> 
Suck it and see ...

The two drives look like they report the same User Capacity, ie 6 
terabytes "exactly". They should therefore be drop-in replacements for 
each other.

And I find that eg when I partition a drive it reports unusable space at 
the end - you should find that any lost space will get lost there ...

And are you using sdk, or sdk1, as your block device to add to the 
array? While I'd be a little surprised, is it possible the disk drives 
are reporting different geometries and messing up fdisk's default 
partition sizes? Try copying the partition geometry exactly from sdi to 
sdk using sector numbers ...

Plus they're both Seagates (Ironwolf, good choice :-), so there really 
shouldn't be a problem. The only odd thing I notice is your new drive 
seems to have an older firmware?

Cheers,
Wol

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Roman Mamedov]

On Thu, 17 Mar 2022 20:08:55 -0700
Marc MERLIN <marc@merlins.org> wrote:

> old drive:
> Device Model:     ST6000VN0041-2EL11C
> Serial Number:    ZA18WX4T
> LU WWN Device Id: 5 000c50 0a47d527a
> Firmware Version: SC61
> User Capacity:    6,001,175,126,016 bytes [6.00 TB]
> Sector Sizes:     512 bytes logical, 4096 bytes physical
> 
>    8      128 5860522584 sdi
>    8      129 5860521543 sdi1
> 
> 
> new drive:
> Device Model:     ST6000VN001-2BB186
> Serial Number:    ZR118A1Y
> LU WWN Device Id: 5 000c50 0dba1b3c0
> Firmware Version: SC60
> User Capacity:    6,001,175,126,016 bytes [6.00 TB]
> Sector Sizes:     512 bytes logical, 4096 bytes physical
> 
>    8      160 5860522580 sdk
>    8      161 5860521536 sdk1
> 
> New drive is 4 sectors shorter, so I assume I can't use it as a replacement in my md5
> array because it's 4 sectors too short, or does swraid5 not need the last few sectors
> of a drive?
> 
> Looks like formatting as MDR won't help, I'm still 4 sectors short.

Check "Used Dev Size" in "mdadm --detail" of your array. I suppose that is how
much (at least) it actually needs from any new member to be suitable for the
array.

If you find it needs more than the size of sdk1, as an emergency measure you
could wipe off the partition table and add the entire sdk as the array member.
While usually not recommended, if you don't boot other operating systems on
the same machine (that could see the "raw" drive and mess with it), it should
not cause any problem.

However there should not be such size difference in the first place, check
your dmesg if drive detection messages report "HPA", and/or check with "hdparm
-N" if there's this HPA enabled, cutting off a portion of the drive at the end.

-- 
With respect,
Roman

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Marc MERLIN]

On Fri, Mar 18, 2022 at 05:30:07PM +0500, Roman Mamedov wrote:
> Check "Used Dev Size" in "mdadm --detail" of your array. I suppose that is how
> much (at least) it actually needs from any new member to be suitable for the
> array.
 
Thank you, it indeed uses a bit less than what was available on the
first drives, so it looks like I'm set
     Used Dev Size : 5860390400 (5588.90 GiB 6001.04 GB)
5860521536-5860390400
131136

> If you find it needs more than the size of sdk1, as an emergency measure you
> could wipe off the partition table and add the entire sdk as the array member.

Yeah, I thought of that, just don't really like it, and not sure if
mdadm -can looks for raw drives in addition to partitions

> However there should not be such size difference in the first place, check
> your dmesg if drive detection messages report "HPA", and/or check with "hdparm
> -N" if there's this HPA enabled, cutting off a portion of the drive at the end.
 

[622993.224113] scsi 11:0:0:0: Direct-Access     Seagate  FA GoFlex Desk   0307 PQ: 0 ANSI: 6
[622993.255658] sd 11:0:0:0: Attached scsi generic sg14 type 0
[622993.273983] sd 11:0:0:0: [sdk] 1465130645 4096-byte logical blocks: (6.00 TB/5.46 TiB)
[622993.300108] sd 11:0:0:0: [sdk] Write Protect is off
[622993.315060] sd 11:0:0:0: [sdk] Mode Sense: 43 00 00 00
[622993.316103] sd 11:0:0:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[622993.392979] sd 11:0:0:0: [sdk] Attached SCSI disk
gargamel:/dev# hdparm -N /dev/sdk
/dev/sdk:
 max sectors   = 11721045168/11721045168, HPA is disabled

On Fri, Mar 18, 2022 at 01:07:49PM -0500, Roger Heflin wrote:
> From the above your partition table seems to be starting at 1044.  There
> are reasons to attempt to start on a boundary, but using a value like 1024
> instead of 1044 should also work just fine and that will give you 20 more
> sectors.

normal fdisk won't allow that, but I forgot that I can go in expert mode
and force the first sector to a value the regular tool won't allow by
default.

Thanks all for the answers and suggestions.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Wols Lists]

On 19/03/2022 04:10, Marc MERLIN wrote:
>> If you find it needs more than the size of sdk1, as an emergency measure you
>> could wipe off the partition table and add the entire sdk as the array member.

> Yeah, I thought of that, just don't really like it, and not sure if
> mdadm -can looks for raw drives in addition to partitions
> 
mdadm has absolutely no trouble with that at all. All it cares about is 
if something is a block device - if it finds an mdadm signature at the 
start of a block device it will use it.

The problem is the eejits out there who assume that all physical drives 
must be partitioned. And we know from experience that there are eejits 
out there who assume that any drive without an MBR or GPT just *must* be 
unused and it's *perfectly* *okay* to write said MBR or GPT *without* 
*asking*. Just trashing your mdadm (or lvm, or whatever yada ydad) 
signature in the process.

It's not common, but we do get calls to recover arrays where the 
signature has gone missing because the owner made the mistake of 
upgrading their system (Windows, linux, whatever) and the upgrade 
stomped all over the array.

Cheers,
Wol

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Roman Mamedov]

On Fri, 18 Mar 2022 17:30:07 +0500
Roman Mamedov <rm@romanrm.net> wrote:

> However there should not be such size difference in the first place

If we look closely though, there actually doesn't appear to be:

> On Thu, 17 Mar 2022 20:08:55 -0700
> Marc MERLIN <marc@merlins.org> wrote:
> 
> > old drive:
> > User Capacity:    6,001,175,126,016 bytes [6.00 TB]
> > 
> > new drive:
> > User Capacity:    6,001,175,126,016 bytes [6.00 TB]

As for...

> >    8      128 5860522584 sdi
> >    8      129 5860521543 sdi1
> >
> >    8      160 5860522580 sdk
> >    8      161 5860521536 sdk1

Which tool returns this output?

What do you get for 

  blockdev --getsize64 /dev/sdi
  blockdev --getsize64 /dev/sdk

If this returns the same size for both, wipe a few first MB the new drive with
zeroes using dd, and try a different partitioning tool.

-- 
With respect,
Roman

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Marc MERLIN]

On Sat, Mar 19, 2022 at 10:14:16AM +0000, Wols Lists wrote:
> mdadm has absolutely no trouble with that at all. All it cares about is if
> something is a block device - if it finds an mdadm signature at the start of
> a block device it will use it.
 
I think I remember back in the day that auto joining of raid1 arrays by
the kernel at boot time so that you could then mount the filesystem as
root, only worked if it was on partitions. That was a long time ago
though.

On Sat, Mar 19, 2022 at 03:45:59PM +0500, Roman Mamedov wrote:
> As for...
> 
> > >    8      128 5860522584 sdi
> > >    8      129 5860521543 sdi1
> > >
> > >    8      160 5860522580 sdk
> > >    8      161 5860521536 sdk1
> 
> Which tool returns this output?
 
cat /proc/partitions

> What do you get for 
> 
>   blockdev --getsize64 /dev/sdi
>   blockdev --getsize64 /dev/sdk

gargamel:/dev# blockdev --getsize64 /dev/sdi
6001175126016
gargamel:/dev# blockdev --getsize64 /dev/sdk
6001175121920

> If this returns the same size for both, wipe a few first MB the new drive with
> zeroes using dd, and try a different partitioning tool.

Good suggestion, but the drives indeed seem different, very slightly so.
Thankfully not enough to matter for my mdadm array it seems, so that's
good news

Thanks to all for the help
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Nix]

On 19 Mar 2022, Wols Lists uttered the following:

> On 19/03/2022 04:10, Marc MERLIN wrote:
>>> If you find it needs more than the size of sdk1, as an emergency measure you
>>> could wipe off the partition table and add the entire sdk as the array member.
>
>> Yeah, I thought of that, just don't really like it, and not sure if
>> mdadm -can looks for raw drives in addition to partitions
>> 
> mdadm has absolutely no trouble with that at all. All it cares about is if something is a block device - if it finds an mdadm
> signature at the start of a block device it will use it.
>
> The problem is the eejits out there who assume that all physical
> drives must be partitioned. And we know from experience that there are
> eejits out there who assume that any drive without an MBR or GPT just
> *must* be unused and it's *perfectly* *okay* to write said MBR or GPT
> *without* *asking*. Just trashing your mdadm (or lvm, or whatever yada
> ydad) signature in the process.

... and we know that some of the eejits out there write EFI firmware :(
and some of them blow away things like this on boot, on resume from
suspend, 

> It's not common

... thank goodness.

-- 
NULL && (void)

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Tom Mitchell]

On Thu, Mar 17, 2022 at 8:41 PM Marc MERLIN <marc@merlins.org> wrote:
>
> old drive:
...
>    8      128 5860522584 sdi
>    8      129 5860521543 sdi1
...
> new drive:
> Device Model:     ST6000VN001-2BB186
>    8      160 5860522580 sdk
>    8      161 5860521536 sdk1
>
> New drive is 4 sectors shorter, so I assume
...

The one option I did not see is resizing the filesystem (smaller) to
make an easy match.

In this case there is a second new disk big enough to back up all the
data so that is
the first task.

Shrinking a filesystem (resize2fs) does have some risks so backup first.

It is sort of nice to leave modest unused space when setting up a new disk.

Also back up to the new disk formatted correctly then invert the question and
add the old disk with matched size partitions to mirror to.

It all depends on the backup strategy. Clonezilla?


-- 

          T o m    M i t c h e l l  (on NiftyEgg[.]com )

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Wols Lists]

On 26/03/2022 18:01, Tom Mitchell wrote:
> Shrinking a filesystem (resize2fs) does have some risks so backup first.
> 
> It is sort of nice to leave modest unused space when setting up a new disk.
> 
> Also back up to the new disk formatted correctly then invert the question and
> add the old disk with matched size partitions to mirror to.
> 
> It all depends on the backup strategy. Clonezilla?

If they are the same size or the backup disk is larger, I'd just use dd 
on an unmounted filesystem!

Cheers,
Wol

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[d tbsky]

Marc MERLIN <marc@merlins.org>
> New drive is 4 sectors shorter, so I assume I can't use it as a replacement in my md5
> array because it's 4 sectors too short, or does swraid5 not need the last few sectors
> of a drive?

    4 sectors shorter are common for external hard drives (sold from
seagate,toshiba,etc). if you attach the drive to internal sata port or
normal usb enclosure(not sold from disk vendors),you will get back
normal disk size. so maybe you want to check what connect to your
disk.

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Marc MERLIN]

On Sun, Mar 27, 2022 at 04:40:38PM +0800, d tbsky wrote:
> Marc MERLIN <marc@merlins.org>
> > New drive is 4 sectors shorter, so I assume I can't use it as a replacement in my md5
> > array because it's 4 sectors too short, or does swraid5 not need the last few sectors
> > of a drive?
> 
>     4 sectors shorter are common for external hard drives (sold from
> seagate,toshiba,etc). if you attach the drive to internal sata port or
> normal usb enclosure(not sold from disk vendors),you will get back
> normal disk size. so maybe you want to check what connect to your
> disk.

OMG, using a USB adapter eats 4 setors? I had no idea...
Sure enough, I did connect the drives via USB for diagnosis since I had
run out of sata ports.

Thanks for the hint.
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/  

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Chris Murphy]

On Sun, Mar 27, 2022 at 8:05 PM Marc MERLIN <marc@merlins.org> wrote:
>
> OMG, using a USB adapter eats 4 setors? I had no idea...
> Sure enough, I did connect the drives via USB for diagnosis since I had
> run out of sata ports.

None of my SATA-USB enclosures behave this way. But what it does do is
mask (lie) the true physical sector size, claiming it's 512 bytes
instead of 4096 bytes.

I've also seen enclosures turn a 512 byte logical sector drive into a
4096 byte logical sector drive, which is a real PITA because it
completely messes up the GPT. My understanding of the logic of GPT
format is atomic modifications that are crash safe (that's up to the
tool to do correctly too but the spec has a pretty specific order each
sector is supposed to be written), but in practice this is another
PITA for a significant minority of users who end up putting an
internal drive into an enclosure and bam - can't read the drive at
all.

It'd be kinda neat if this condition could be detected and have a
device mapper target setup to deal with it.



-- 
Chris Murphy

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[d tbsky]

Chris Murphy <lists@colorremedies.com>
> None of my SATA-USB enclosures behave this way. But what it does do is
> mask (lie) the true physical sector size, claiming it's 512 bytes
> instead of 4096 bytes.

normal sata-usb should be ok. seagate/toshiba sata-usb will eat
sectors. some gigabyte dual-bios motherboards will eat sectors (with
disk HPA function).
In the early days different vendors made different capacity harddisks.
But at some moment, maybe 250GB or 500GB, suddenly every vendor made
the same capacity harddisks. It's a mystery to me. who decides the
disk sector numbers? why 2TB disk capacity is not double 1TB, 8TB not
double 4TB? capacity is different for sd-card and ssd, but harddisk
capacity seems normalized at some time.

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[Andy Smith]

Hi,

On Tue, Mar 29, 2022 at 09:55:40AM +0800, d tbsky wrote:
> In the early days different vendors made different capacity harddisks.
> But at some moment, maybe 250GB or 500GB, suddenly every vendor made
> the same capacity harddisks. It's a mystery to me. who decides the
> disk sector numbers?

There is a standard called IDEMA LBA1-03:

    http://www.idema.org/wp-content/downloads/2169.pdf

This says that a certain "marketing capacity" (i.e. when the drive
product description says "2TB" or whatever) will equal an exact
number of 512 or 4096 byte sectors.

It's been over a decade since I personally saw a SATA, SAS or NVMe
drive that did not obey this, so I've felt comfortable that I could
replace drives from one vendor with another without having to worry
about a few sectors different size here or there.

However, I did see people on this and other mailing lists such as
zfs-discuss saying they were still seeing drives that did not comply
with IDEMA LBA1-03 for capacity as recently as last year, so
apparently I did not look hard enough or have just been lucky.

If you want to play at home the formula in IDEMA LBA1-03 boils down
to:

    ($GB * 1000194048) + 10838016 bytes

using powers of ten definitions for "GB", so anything calling itself
a "2TB" drive should be exactly:

    (2000 * 1000194048) + 10838016

    = 2,000,398,934,016 bytes

Capacity presented to the OS might be lower of course if there is
hardware meddling as you mention.

Again, I've not personally seen anything recently that doesn't obey
this, but people have shown me things that don't, so it's
apparently still a thing.

Cheers,
Andy

Re: new drive is 4 sectors shorter, can it be used for swraid5 array?

[d tbsky]

Andy Smith <andy@strugglers.net>
> There is a standard called IDEMA LBA1-03:
>
>     http://www.idema.org/wp-content/downloads/2169.pdf
>
> This says that a certain "marketing capacity" (i.e. when the drive
> product description says "2TB" or whatever) will equal an exact
> number of 512 or 4096 byte sectors.

  thanks a lot for the information!  now I understand what happened to
my disk drives :)