What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Hillel Lubman]

[-- Attachment #1.1: Type: text/plain, Size: 829 bytes --]

Hi.

Looking around I saw some references that it's preferable to use sector size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e) advanced format hard drives and also some mentions of making that a default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1) used with current Debian testing (Linux 3.16.0) still uses 512 B sectors by default.

Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).

Can you please also add this topic to the XFS FAQ (http://www.xfs.org/index.php/XFS_FAQ[1]), since I couldn't find any information about it there.

Thanks!

Hillel Lubman.

--------
[1] http://www.xfs.org/index.php/XFS_FAQ

[-- Attachment #1.2: Type: text/html, Size: 2830 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Stan Hoeppner]

On 01/04/2015 06:56 PM, Hillel Lubman wrote:
...
> Looking around I saw some references that it's preferable to use sector
> size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e)
> advanced format hard drives and also some mentions of making that a
> default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1)
> used with current Debian testing (Linux 3.16.0) still uses 512 B sectors
> by default.
...
> Can you please clarify what after all is the recommended sector size for
> such drives and why isn't it a default in mkfs.xfs (since supposedly
> defaults are generally recommended optimal settings unless you have some
> special use case).
...

XFS sectsz is unimportant with these drives.  What matters is that any
partitions you create start and end on 4KB boundaries.  This will
prevent adjacent hardware sector RMW internal to the drive.  XFS writes
in 4KB filesystem blocks on Linux.  As long as the fsblocks are aligned
to the 4KB hardware sectors there's nothing more you can do to avoid
performance penalties with these drives.

Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Hillel Lubman]

[-- Attachment #1.1: Type: text/plain, Size: 749 bytes --]

On Sunday, January 04, 2015 21:52:51 Stan Hoeppner wrote:
> On 01/04/2015 06:56 PM, Hillel Lubman wrote:
> ...
> > when creating XFS partitions on hybrid (512e)
> ...
> > Can you please clarify what after all is the recommended sector size for
> > such drives 
> ...
> 
> XFS sectsz is unimportant with these drives.  What matters is that any
> partitions you create start and end on 4KB boundaries. 

Thanks. So there shouldn't be any negative impact either if sectz is set to 4 KB? In one particular case I experience some perceived slowness on 2 TB WD drive (and I created XFS partition on that drive using sectz as 4 KB). I didn't test however how it compares to default settings on the same drive which would set it to 512 B.

Regards,
Hillel.

[-- Attachment #1.2: Type: text/html, Size: 3390 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Stan Hoeppner]

On 01/05/2015 08:23 PM, Hillel Lubman wrote:
> On Sunday, January 04, 2015 21:52:51 Stan Hoeppner wrote:
> 
>> On 01/04/2015 06:56 PM, Hillel Lubman wrote:
> 
>> ...
> 
>> > when creating XFS partitions on hybrid (512e)
> 
>> ...
> 
>> > Can you please clarify what after all is the recommended sector size for
> 
>> > such drives
> 
>> ...
> 
>>
> 
>> XFS sectsz is unimportant with these drives. What matters is that any
> 
>> partitions you create start and end on 4KB boundaries.
> 
>  
> 
> Thanks. So there shouldn't be any negative impact either if sectz is set
> to 4 KB? 

Again, as long as your partitions are 4KB aligned.

> In one particular case I experience some perceived slowness on
> 2 TB WD drive (and I created XFS partition on that drive using sectz as
> 4 KB). 

Likely due to a misaligned partition.  In this case many XFS IOs are
going to cause RMW in two adjacent hardware (4KB) disk sectors as each
XFS block overlaps two sectors.  This may significantly hamper drive
performance.

> I didn't test however how it compares to default settings on the
> same drive which would set it to 512 B.

As long as your underlying partition is 4KB aligned the only advantage
you'll likely see with 4B sectsz is a little faster log IO.  So for non
metadata heavy workloads you won't see any difference between 512B and
4KB sectsz.

Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Hillel Lubman]

[-- Attachment #1.1: Type: text/plain, Size: 1361 bytes --]

On Monday, January 05, 2015 21:21:25 Stan Hoeppner wrote:
> 
> > I didn't test however how it compares to default settings on the
> > same drive which would set it to 512 B.
> 
> As long as your underlying partition is 4KB aligned the only advantage
> you'll likely see with 4B sectsz is a little faster log IO.  So for non
> metadata heavy workloads you won't see any difference between 512B and
> 4KB sectsz.
> 
> Stan

It's actually created by gparted with 1 MB alignment, so it's implicitly 4 KB aligned as well. I guess something else is causing the slowness or it's just subjective. Raw I/O performance test looks normal with hdparm.

Here is what gdisk reports (third partition is the XFS one):

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): ...
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 4205 sectors (2.1 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          534527   260.0 MiB   EF00  
   2          534528         1763327   600.0 MiB   0700  
   3         1763328      3904931839   1.8 TiB     0700  
   4      3904931840      3907026943   1023.0 MiB  8200

Regards,
Hillel.


[-- Attachment #1.2: Type: text/html, Size: 6593 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Matthias Schniedermeyer]

On 04.01.2015 21:52, Stan Hoeppner wrote:
> On 01/04/2015 06:56 PM, Hillel Lubman wrote:
> ...
> > Looking around I saw some references that it's preferable to use sector
> > size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e)
> > advanced format hard drives and also some mentions of making that a
> > default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1)
> > used with current Debian testing (Linux 3.16.0) still uses 512 B sectors
> > by default.
> ...
> > Can you please clarify what after all is the recommended sector size for
> > such drives and why isn't it a default in mkfs.xfs (since supposedly
> > defaults are generally recommended optimal settings unless you have some
> > special use case).
> ...
> 
> XFS sectsz is unimportant with these drives.  What matters is that any
> partitions you create start and end on 4KB boundaries.  This will
> prevent adjacent hardware sector RMW internal to the drive.  XFS writes
> in 4KB filesystem blocks on Linux.  As long as the fsblocks are aligned
> to the 4KB hardware sectors there's nothing more you can do to avoid
> performance penalties with these drives.

Not in my experience. My first (and lying about it) AF HDD had horrible 
write performance until i re'mkfs.xfs'ed it with 4k sector-size. And yes 
the aligned was 4k.

But that was many years ago and i don't know if newer XFS (IIRC that was 
before delay-log) would behave better. But as all of my storage after 
the first AF HDD was either an AF HDD or a SSD i have been formating 
everything with 4k sector-size for years.




-- 

Matthias

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Eric Sandeen]

On 1/4/15 6:56 PM, Hillel Lubman wrote:
> Hi.


> Can you please clarify what after all is the recommended sector size
> for such drives and why isn't it a default in mkfs.xfs (since
> supposedly defaults are generally recommended optimal settings unless
> you have some special use case).

It is indeed the default.  

[root@sandeen ~]# blockdev --getss --getpbsz /dev/sde
512
4096
[root@sandeen ~]# mkfs.xfs /dev/sde
meta-data=/dev/sde               isize=256    agcount=1, agsize=4097 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=4097, imaxpct=25
         =                       sunit=1      swidth=8 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=1424, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


however, some drives lie about these sizes, and then mkfs.xfs can't know.
Try the blockdev command above to see.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Chris Murphy]

On Tue, Jan 6, 2015 at 10:02 AM, Eric Sandeen <sandeen@sandeen.net> wrote:
> however, some drives lie about these sizes, and then mkfs.xfs can't know.
> Try the blockdev command above to see.

blockdev and parted seem to get this wrong for a device for which
smartctl and hdparm get correct


smartctl -a reports:
Sector Sizes:     512 bytes logical, 4096 bytes physical
hdparm -I reports:
Logical  Sector size:                   512 bytes
Physical Sector size:                  4096 bytes

parted reports:
Sector size (logical/physical): 512B/512B

# blockdev --getss /dev/sdc
512
# blockdev --getpbsz /dev/sdc
512

And thus xfs_info on this device says sectsz=512 since I formatted
with defaults.



-- 
Chris Murphy

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Eric Sandeen]

On 1/6/15 1:05 PM, Chris Murphy wrote:
> On Tue, Jan 6, 2015 at 10:02 AM, Eric Sandeen <sandeen@sandeen.net> wrote:
>> however, some drives lie about these sizes, and then mkfs.xfs can't know.
>> Try the blockdev command above to see.
> 
> blockdev and parted seem to get this wrong for a device for which
> smartctl and hdparm get correct

I don't think they get it wrong, they are just reporting what the
drive says over that interface.

I ... don't actually know where smartctl/hdparm get the values, vs.
the values exported to blockdev.  Is the drive reporting different
values over different query interfaces?  Hrm.

mkfs.xfs uses the same interface as used by blockdev.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Chris Murphy]

On Tue, Jan 6, 2015 at 12:23 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> On 1/6/15 1:05 PM, Chris Murphy wrote:
>> On Tue, Jan 6, 2015 at 10:02 AM, Eric Sandeen <sandeen@sandeen.net> wrote:
>>> however, some drives lie about these sizes, and then mkfs.xfs can't know.
>>> Try the blockdev command above to see.
>>
>> blockdev and parted seem to get this wrong for a device for which
>> smartctl and hdparm get correct
>
> I don't think they get it wrong, they are just reporting what the
> drive says over that interface.

The problem may be that the drive isn't being queried, but rather the
bridge chipset involved. All drives are AF drives in USB 3 enclosures.
But somehow hdparm and smartctl are getting actual drive info from the
drive through this interface; while the kernel appears to be fooled by
bridge.

I don't immediately have a way to directly connect any to SATA. I will
bet dollars to donuts this is what's going on though, because I know
one of these drives when it was connected via SATA was reported by
parted as 512B/4096B but currently it isn't.

-- 
Chris Murphy

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Chris Murphy]

What's sectsz= used for? Historically this would be sectsz=512,
bsize=4096 before AF hard drives appeared. So is sectsz used for
optimizing sub blocksize changes? e.g. if only 50 bytes needs
changing, the fs doesn't need to read modify and write the entire 4096
block, just change the affected sector?

If that's true, then sectsz=512 for a 4096 physical sector drive would
cause a lot of in-drive RMW, depending on the workdlow. On older AF
drives this can be a huge penalty hit.


Chris Murphy

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Hillel Lubman]

[-- Attachment #1.1: Type: text/plain, Size: 1052 bytes --]

On Tuesday, January 06, 2015 11:02:09 Eric Sandeen wrote:
> On 1/4/15 6:56 PM, Hillel Lubman wrote:
> > Hi.
> 
> 
> > Can you please clarify what after all is the recommended sector size
> > for such drives and why isn't it a default in mkfs.xfs (since
> > supposedly defaults are generally recommended optimal settings unless
> > you have some special use case).
> 
> It is indeed the default.  
> 
> [root@sandeen ~]# blockdev --getss --getpbsz /dev/sde
> 512
> 4096
> ...
> 
> however, some drives lie about these sizes, and then mkfs.xfs can't know.
> Try the blockdev command above to see.
> 

I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
512
512

However hdparm -I reports:
Logical  Sector size:    512 bytes
Physical Sector size:     4096 bytes

So is it still worthwhile making sectsz 4 KB explicitly instead of using mkfs.xfs default in such case? And on a side note, since there is a more reliable way to figure that info out, may be mkfs.xfs should rather use that?

Regards,
Hillel.

[-- Attachment #1.2: Type: text/html, Size: 6151 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Eric Sandeen]

On 1/6/15 11:17 PM, Hillel Lubman wrote:

> I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
> 
> 512
> 512

what kind of drive is this; is it usb, SATA, or ?

> However hdparm -I reports:
> 
> Logical Sector size: 512 bytes
> 
> Physical Sector size: 4096 bytes
> 
> So is it still worthwhile making sectsz 4 KB explicitly instead of
> using mkfs.xfs default in such case? And on a side note, since there
> is a more reliable way to figure that info out, may be mkfs.xfs
> should rather use that?
>

a) probably can't hurt
b) probably doesn't matter ;)

c) not sure why there are 2 different answers.  Knowing what kind of drive
it is would be good; mkfs.xfs just uses whatever blkid says.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Eric Sandeen]

On 1/6/15 1:55 PM, Chris Murphy wrote:
> What's sectsz= used for? Historically this would be sectsz=512,
> bsize=4096 before AF hard drives appeared. So is sectsz used for
> optimizing sub blocksize changes? e.g. if only 50 bytes needs
> changing, the fs doesn't need to read modify and write the entire 4096
> block, just change the affected sector?

Nope, filesystem data blocks are always fully written, but the sector
size is i.e. the minimum _log_ IO size.

+/*
+ * The xfs_buftarg contains 2 notions of "sector size" -
+ *
+ * 1) The metadata sector size, which is the minimum unit and
+ *    alignment of IO which will be performed by metadata operations.
+ * 2) The device logical sector size
+ *
+ * The first is specified at mkfs time, and is stored on-disk in the
+ * superblock's sb_sectsize.
+ *
+ * The latter is derived from the underlying device, and controls direct IO
+ * alignment constraints.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Hillel Lubman]

[-- Attachment #1.1: Type: text/plain, Size: 672 bytes --]

On Tuesday, January 06, 2015 23:30:26 Eric Sandeen wrote:
> On 1/6/15 11:17 PM, Hillel Lubman wrote:
> 
> > I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
> > 
> > 512
> > 512
> 
> what kind of drive is this; is it usb, SATA, or ?
> 
> c) not sure why there are 2 different answers.  Knowing what kind of drive
> it is would be good; mkfs.xfs just uses whatever blkid says.
> 

One is a recent portable WD USB3 drive (elements) and another is a regular 3.5" internal AF WD black in an enclosure (also attached through USB3). I didn't recently test the case of a drive attached through regular SATA.

Regards,
Hillel.

[-- Attachment #1.2: Type: text/html, Size: 3609 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Chris Murphy]

On Tue, Jan 6, 2015 at 10:31 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> On 1/6/15 1:55 PM, Chris Murphy wrote:
>> What's sectsz= used for? Historically this would be sectsz=512,
>> bsize=4096 before AF hard drives appeared. So is sectsz used for
>> optimizing sub blocksize changes? e.g. if only 50 bytes needs
>> changing, the fs doesn't need to read modify and write the entire 4096
>> block, just change the affected sector?
>
> Nope, filesystem data blocks are always fully written, but the sector
> size is i.e. the minimum _log_ IO size.

So maybe the people reporting performance problems with sectsz=512 on
AF drives compared to manually specifying 4096 bytes, have a high
metadata workload with older AF drives and are getting lot of in-drive
RMW...


-- 
Chris Murphy

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?

[Chris Murphy]

On Tue, Jan 6, 2015 at 10:36 PM, Hillel Lubman <shtetldik@gmail.com> wrote:

> One is a recent portable WD USB3 drive (elements) and another is a regular
> 3.5" internal AF WD black in an enclosure (also attached through USB3). I
> didn't recently test the case of a drive attached through regular SATA.

A common theme emerges, so far 6 for 6 drives behaving this way are in
USB enclosures. At least one vendor ships an external drive (drive
inside an enclosure) that causes parted to report 4096B/4096B
logical/physical sectors. The drive inside is actually 512e so it's
512B/4096B, and the result of this is that partitioning/formatting it
inside the enclosure means it's completely unreadable outside the
enclosure and vice versa. So cute and adorable.


-- 
Chris Murphy

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs