[linux-lvm] Data alignment

[linux-lvm] Data alignment

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I have been getting warnings from mke2fs lately that my logical
volumes are not aligned properly.  I am using a raid5 array as a pv
and it has a 1.5 MB stripe width, so logical volumes should be aligned
to that, but it appears lvm is not doing this.

It looks like you can specify this alignment at pvcreate time, but I
don't think I did and it doesn't seem to have been picked up
automatically.  Is there a way to change this without rebuilding the vg?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJRGliwAAoJEJrBOlT6nu75u38IANP/fcPpeXdzymKzDBmKFjJ/
hVSR0Ig8lcym3nbL9wCz/PIGRrta5j9M2lVrIoMP68iTcywqsyON7f+iPN3f20MO
v3Vjm0kRET4rw8inJE2g+6o5RlOAn+45vN6G6LtkncPFEo8MhxwHO2EMno3nql7Y
Ds5eFd9OjK8BkpUaMjzUJR61of5SL32HXTAm7mVMXSzV1IIzFvSMq8P5bzI6GGnx
mtoMqcND64+5AvU/FBnmB12k0MVNbnpLRvj1CZnhM8rrHu5cN921O1zK3M8rZ7sb
vk9GalquS+AuELBwLOiY2t6a5ebW2aWB4Yi2docdOjudeN1y7LrvgmNmhIbxE+k=
=C45A
-----END PGP SIGNATURE-----

Re: [linux-lvm] Data alignment

[Mike Snitzer]

On Tue, Feb 12 2013 at  9:58am -0500,
Phillip Susi <psusi@ubuntu.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I have been getting warnings from mke2fs lately that my logical
> volumes are not aligned properly.  I am using a raid5 array as a pv
> and it has a 1.5 MB stripe width, so logical volumes should be aligned
> to that, but it appears lvm is not doing this.
> 
> It looks like you can specify this alignment at pvcreate time, but I
> don't think I did and it doesn't seem to have been picked up
> automatically.  Is there a way to change this without rebuilding the vg?

Please provide the version of lvm2 that you're using.

For MD raid5, lvm2 should pick up the alignment without any manual flags
being required.  The manual flags are only required for users who have
hardware raid where the minimum_io_size and optimal_io_size isn't
populated in /sys/block/<dev>/queue/

But MD does populate those and lvm2 should definitely consume them when
laying out the data area of a PV.  You should be able to see what
pvcreate determines the {minimum,optimal}_io_size to be by running:
pvcreate -vvvv <args>

But no, after the data area is established there is no way to change the
alignment after the fact.

Re: [linux-lvm] Data alignment

[Stuart D. Gathman]

On Fri, 19 Mar 2010, Phillip Susi wrote:

> I have been trying to get my logical volume to be aligned to a 512 KB
> boundary on my SSD so it lines up with the flash erase block.  I thought
> I could do this with pvcreate --dataalign 1024, but this caused the data
> to be aligned to a 1024 sector boundary within the partition, which
> itself starts on sector 63.  I then thought the --dataalignoffsest
> switch would help, but specifying 63 there caused the data to be moved
> to sector 1087.  I tried giving it -63, but was told that a negative
> number is not allowed.  Why not?  How can I get the data to start on
> sector 1024-63 of the partition so it ends up on pysical sector 1024 of
> the disk?

You can have "negative" offsets.  Just subtract from the modulus.  In your
case that would be 1024-63.  I.e. --dataalignoffset 961

If the PV is not removeable, you can skip the partition table and use the
entire device for the PV - leaving 63 more sectors for metadata.

If the PV is removeable, remember you need vgchange -ay vgname after 
attaching, and vgchange -an vgname before removing.  (I would suggest
leaving the partition table if removeable so that other systems will
recognize the partition label.)

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] Data alignment

[Luca Berra]

On Fri, Mar 19, 2010 at 02:54:52PM -0400, Phillip Susi wrote:
>On 3/19/2010 2:36 PM, Eugene Vilensky wrote:
>> Align the partition?
>
>Then I waste 512k on the MBR, and another 512k on the pv header.  I'd
>like to avoid that if possible and it seems like --dataalignoffset
>should let me do that, but it doesn't accept negative numbers.
then, just do not create the partition and use lvm on the whole disk :P

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: [linux-lvm] Data alignment

[Mike Snitzer]

On Fri, Mar 19 2010 at  6:21pm -0400,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Fri, Mar 19 2010 at  5:27pm -0400,
> Phillip Susi <psusi@cfl.rr.com> wrote:
> 
> > On 3/19/2010 3:25 PM, Mike Snitzer wrote:
> > > Any reason why you'd like to use DOS partitioning (first primary
> > > partition starting at 63rd sector)?
> > 
> > Convention.  I am looking into using GPT instead though.
> > 
> > > Anyway, if you have a recent kernel (e.g. 2.6.33) you'll be in good
> > > shape regardless.  If you create a partition on the device (using DOS
> > > partitions) the kernel _should_ be aware of where the partition starts
> > > and tell you how far you'd have to shift the start of your LVM2 PV data
> > > area to get it perfectly aligned relative to the underlying physical
> > > block size.  Check for example:
> > > # cat /sys/block/sda/sda1/alignment_offset
> > > 
> > > But if your device is using 512b physical_sector_size you'll just have a
> > > 0 for alignment_offset.  Check physical_block_size with:
> > > # cat /sys/block/sda/queue/physical_block_size
> > 
> > 0 alignment, physical block size 512.
> > 
> > > Also verify that your SSD device is naturally aligned (aka
> > > alignment_offset=0); I'd wager it is naturally aligned:
> > > # cat /sys/block/sda/alignment_offset
> > 
> > Kernel thinks so, and based on performance tests it appears so.
> > 
> > > All said, even if you have an older kernel, to manually get what you
> > > want (shift start to account for DOS partition at 63rd sector, align PV
> > > pe_start on a 512K boundary), please try:
> > > # pvcreate --dataalignmentoffset 512b --dataalignment 512K ...
> > 
> > Won't that just add one sector to the start, placing it at sector 1025?
> >  How does an alignment offset of 1 sector account for the partition
> > starting on sector 63?
> 
> The --dataalignmentoffset acts as padding.  I was focused on getting you
> to a power of 2 start (sector 64).  Then from there --dataalignment
> governs the start boundary for the PV data area.
> 
> So you're right, 64*512b + 512K isn't a multiple of 512K :)
> 
> This will get you what you want:
> # pvcreate --dataalignmentoffset 512b --dataalignment 480K /dev/sda1

Alternatively you could do (as long as sysfs has alignment_offset=0):
# pvcreate --dataalignment 480.5K /dev/sda1

or

# pvcreate --dataalignment 961S /dev/sda1

Re: [linux-lvm] Data alignment

[Mike Snitzer]

On Fri, Mar 19 2010 at  5:27pm -0400,
Phillip Susi <psusi@cfl.rr.com> wrote:

> On 3/19/2010 3:25 PM, Mike Snitzer wrote:
> > Any reason why you'd like to use DOS partitioning (first primary
> > partition starting at 63rd sector)?
> 
> Convention.  I am looking into using GPT instead though.
> 
> > Anyway, if you have a recent kernel (e.g. 2.6.33) you'll be in good
> > shape regardless.  If you create a partition on the device (using DOS
> > partitions) the kernel _should_ be aware of where the partition starts
> > and tell you how far you'd have to shift the start of your LVM2 PV data
> > area to get it perfectly aligned relative to the underlying physical
> > block size.  Check for example:
> > # cat /sys/block/sda/sda1/alignment_offset
> > 
> > But if your device is using 512b physical_sector_size you'll just have a
> > 0 for alignment_offset.  Check physical_block_size with:
> > # cat /sys/block/sda/queue/physical_block_size
> 
> 0 alignment, physical block size 512.
> 
> > Also verify that your SSD device is naturally aligned (aka
> > alignment_offset=0); I'd wager it is naturally aligned:
> > # cat /sys/block/sda/alignment_offset
> 
> Kernel thinks so, and based on performance tests it appears so.
> 
> > All said, even if you have an older kernel, to manually get what you
> > want (shift start to account for DOS partition at 63rd sector, align PV
> > pe_start on a 512K boundary), please try:
> > # pvcreate --dataalignmentoffset 512b --dataalignment 512K ...
> 
> Won't that just add one sector to the start, placing it at sector 1025?
>  How does an alignment offset of 1 sector account for the partition
> starting on sector 63?

The --dataalignmentoffset acts as padding.  I was focused on getting you
to a power of 2 start (sector 64).  Then from there --dataalignment
governs the start boundary for the PV data area.

So you're right, 64*512b + 512K isn't a multiple of 512K :)

This will get you what you want:
# pvcreate --dataalignmentoffset 512b --dataalignment 480K /dev/sda1

# pvs --units s -o+pe_start /dev/sda1
  PV         VG   Fmt  Attr PSize     PFree     1st PE 
  /dev/sda1       lvm2 a-   20971457S 20971457S    961S

961*512b + 63*512b = 512K

Re: [linux-lvm] Data alignment

[Phillip Susi]

On 3/19/2010 3:25 PM, Mike Snitzer wrote:
> Any reason why you'd like to use DOS partitioning (first primary
> partition starting at 63rd sector)?

Convention.  I am looking into using GPT instead though.

> Anyway, if you have a recent kernel (e.g. 2.6.33) you'll be in good
> shape regardless.  If you create a partition on the device (using DOS
> partitions) the kernel _should_ be aware of where the partition starts
> and tell you how far you'd have to shift the start of your LVM2 PV data
> area to get it perfectly aligned relative to the underlying physical
> block size.  Check for example:
> # cat /sys/block/sda/sda1/alignment_offset
> 
> But if your device is using 512b physical_sector_size you'll just have a
> 0 for alignment_offset.  Check physical_block_size with:
> # cat /sys/block/sda/queue/physical_block_size

0 alignment, physical block size 512.

> Also verify that your SSD device is naturally aligned (aka
> alignment_offset=0); I'd wager it is naturally aligned:
> # cat /sys/block/sda/alignment_offset

Kernel thinks so, and based on performance tests it appears so.

> All said, even if you have an older kernel, to manually get what you
> want (shift start to account for DOS partition at 63rd sector, align PV
> pe_start on a 512K boundary), please try:
> # pvcreate --dataalignmentoffset 512b --dataalignment 512K ...

Won't that just add one sector to the start, placing it at sector 1025?
 How does an alignment offset of 1 sector account for the partition
starting on sector 63?

Re: [linux-lvm] Data alignment

[Phillip Susi]

On 3/19/2010 3:54 PM, Ray Morris wrote:
>    1 MB "wasted" on a 160,000 MB SDD = 99.999375% efficiency.
> Inversely, that's 0.000625% "wasted", worth roughly 1/5 of
> 1 penny US, or $0.0002.  Your time spent thinking about it - $25.
> Also, 10,000 sysadmins on this list X 2 minutes = 3333 hours X
> $100/hour = $333,333.  I'd burn the 1MB and move on.

Good point ;)

Re: [linux-lvm] Data alignment

[Ray Morris]

> Then I waste 512k on the MBR, and another 512k on the pv header.
> I'd like to avoid that if possible and it seems like --dataalignoffset
> should let me do that, but it doesn't accept negative numbers.

    1 MB "wasted" on a 160,000 MB SDD = 99.999375% efficiency.
Inversely, that's 0.000625% "wasted", worth roughly 1/5 of
1 penny US, or $0.0002.  Your time spent thinking about it - $25.
Also, 10,000 sysadmins on this list X 2 minutes = 3333 hours X
$100/hour = $333,333.  I'd burn the 1MB and move on.
--
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php


On 03/19/2010 01:54:52 PM, Phillip Susi wrote:
> On 3/19/2010 2:36 PM, Eugene Vilensky wrote:
> > Align the partition?
> 
> Then I waste 512k on the MBR, and another 512k on the pv header.  I'd
> like to avoid that if possible and it seems like --dataalignoffset
> should let me do that, but it doesn't accept negative numbers.
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] Data alignment

[Mike Snitzer]

On Fri, Mar 19 2010 at  2:13pm -0400,
Phillip Susi <psusi@cfl.rr.com> wrote:

> I have been trying to get my logical volume to be aligned to a 512 KB
> boundary on my SSD so it lines up with the flash erase block.  I thought
> I could do this with pvcreate --dataalign 1024, but this caused the data
> to be aligned to a 1024 sector boundary within the partition, which
> itself starts on sector 63.  I then thought the --dataalignoffsest
> switch would help, but specifying 63 there caused the data to be moved
> to sector 1087.  I tried giving it -63, but was told that a negative
> number is not allowed.  Why not?  How can I get the data to start on
> sector 1024-63 of the partition so it ends up on pysical sector 1024 of
> the disk?

Which kernel and lvm2 are you using?

I'm assuming you have a recent lvm2 given the flags you're referring
to.  But being precise when asking such questions is helpful.

Any reason why you'd like to use DOS partitioning (first primary
partition starting at 63rd sector)?

Anyway, if you have a recent kernel (e.g. 2.6.33) you'll be in good
shape regardless.  If you create a partition on the device (using DOS
partitions) the kernel _should_ be aware of where the partition starts
and tell you how far you'd have to shift the start of your LVM2 PV data
area to get it perfectly aligned relative to the underlying physical
block size.  Check for example:
# cat /sys/block/sda/sda1/alignment_offset

But if your device is using 512b physical_sector_size you'll just have a
0 for alignment_offset.  Check physical_block_size with:
# cat /sys/block/sda/queue/physical_block_size

Also verify that your SSD device is naturally aligned (aka
alignment_offset=0); I'd wager it is naturally aligned:
# cat /sys/block/sda/alignment_offset

New versions of LVM2 (>= 2.02.51) support detecting alignment attributes
from the kernel.  But the newer LVM2 is the better (there have been a
few related fixes over the LVM2 releases, as recently as 2.02.62).

All said, even if you have an older kernel, to manually get what you
want (shift start to account for DOS partition at 63rd sector, align PV
pe_start on a 512K boundary), please try:
# pvcreate --dataalignmentoffset 512b --dataalignment 512K ...

You'll note that the pe_start may not be where you'd first expect
(e.g. sector 1024-63); but it _should_ be aligned how you'd want it (on
a native 512K boundary).  The reason for this is LVM reserves space at
the beginning of the PV.  Check pe_start with:
# pvs -o+pe_start

Mike

Re: [linux-lvm] Data alignment

[Phillip Susi]

On 3/19/2010 2:36 PM, Eugene Vilensky wrote:
> Align the partition?

Then I waste 512k on the MBR, and another 512k on the pv header.  I'd
like to avoid that if possible and it seems like --dataalignoffset
should let me do that, but it doesn't accept negative numbers.

Re: [linux-lvm] Data alignment

[Eugene Vilensky]

On Fri, Mar 19, 2010 at 1:13 PM, Phillip Susi <psusi@cfl.rr.com> wrote:
> I have been trying to get my logical volume to be aligned to a 512 KB
> boundary on my SSD so it lines up with the flash erase block. �I thought
> I could do this with pvcreate --dataalign 1024, but this caused the data
> to be aligned to a 1024 sector boundary within the partition, which
> itself starts on sector 63. �I then thought the --dataalignoffsest
> switch would help, but specifying 63 there caused the data to be moved
> to sector 1087. �I tried giving it -63, but was told that a negative
> number is not allowed. �Why not? �How can I get the data to start on
> sector 1024-63 of the partition so it ends up on pysical sector 1024 of
> the disk?

Align the partition?

[linux-lvm] Data alignment

[Phillip Susi]

I have been trying to get my logical volume to be aligned to a 512 KB
boundary on my SSD so it lines up with the flash erase block.  I thought
I could do this with pvcreate --dataalign 1024, but this caused the data
to be aligned to a 1024 sector boundary within the partition, which
itself starts on sector 63.  I then thought the --dataalignoffsest
switch would help, but specifying 63 there caused the data to be moved
to sector 1087.  I tried giving it -63, but was told that a negative
number is not allowed.  Why not?  How can I get the data to start on
sector 1024-63 of the partition so it ends up on pysical sector 1024 of
the disk?