[linux-lvm] "Data alignment must not exceed device size."

[linux-lvm] "Data alignment must not exceed device size."

[Richard W.M. Jones]

Recently in Fedora something changed that stops us from creating small
LVs for testing.

An example failure with a 64 MB partitioned disk:

# parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
# lvm pvcreate --force /dev/sda1
  /dev/sda1: Data alignment must not exceed device size.
  Format-specific initialisation of physical volume /dev/sda1 failed.
  Failed to setup physical volume "/dev/sda1".

Obviously if I were to increase the beginning of the partition to
65536 sectors (32 MB) then there wouldn't be much space left for data.

Another possibly related issue happens if I use a 32 MB disk and leave
out the partition table (again, this used to work in Fedora up to recently):

# lvm pvcreate --force /dev/sda
# lvm vgcreate VG /dev/sda
# lvm lvcreate -l 100%FREE -n LV /dev/VG
  Calculated size of logical volume is 0 extents. Needs to be larger.
guestfsd: error:   Calculated size of logical volume is 0 extents. Needs to be larger.

I checked the lvm2 git repo and didn't see any recent changes which I
thought could affect this.

I'm not sure what component is responsible.  I have two very similarly
configured machines where it works with one but not with the other.
The only difference seems to be the kernel:

Working:
  kernel-4.15.0-0.rc8.git0.1.fc28.x86_64
  device-mapper-1.02.146-1.fc28.x86_64
  lvm2-2.02.177-1.fc28.x86_64

Not working:
  kernel-4.16.0-0.rc4.git0.1.fc28.x86_64
  device-mapper-1.02.146-1.fc28.x86_64
  lvm2-2.02.177-1.fc28.x86_64

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

Re: [linux-lvm] "Data alignment must not exceed device size."

[Richard W.M. Jones]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
> Recently in Fedora something changed that stops us from creating small
> LVs for testing.
> 
> An example failure with a 64 MB partitioned disk:
> 
> # parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
> Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
> # lvm pvcreate --force /dev/sda1
>   /dev/sda1: Data alignment must not exceed device size.
>   Format-specific initialisation of physical volume /dev/sda1 failed.
>   Failed to setup physical volume "/dev/sda1".

Interestingly the alignment properties of the virtio-scsi virtual disk
has changed.  On the working system:

==> /sys/block/sda/queue/minimum_io_size <==
512

==> /sys/block/sda/queue/optimal_io_size <==
0

On the new / broken system:

==> /sys/block/sda/queue/minimum_io_size <==
33553920

==> /sys/block/sda/queue/optimal_io_size <==
0

(all other settings were the same).  I suppose this accounts for it,
and it could therefore be a problem with qemu rather than LVM.

Working: qemu-2.11.0-4.fc28.x86_64
Broken:  qemu-2.12.0-0.5.rc1.fc29.x86_64

Alasdair Kergon asked me to run the lvm command with -vvvv and I've
attached those results.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[-- Attachment #2: broken.txt.xz --]
[-- Type: application/x-xz, Size: 3088 bytes --]

[-- Attachment #3: working.txt.xz --]
[-- Type: application/x-xz, Size: 3628 bytes --]

Re: [linux-lvm] "Data alignment must not exceed device size."

[Zdenek Kabelac]

Dne 10.4.2018 v 16:00 Richard W.M. Jones napsal(a):
> On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
>> Recently in Fedora something changed that stops us from creating small
>> LVs for testing.
>>
>> An example failure with a 64 MB partitioned disk:
>>
>> # parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
>> Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
>> # lvm pvcreate --force /dev/sda1
>>    /dev/sda1: Data alignment must not exceed device size.
>>    Format-specific initialisation of physical volume /dev/sda1 failed.
>>    Failed to setup physical volume "/dev/sda1".
> 
> Interestingly the alignment properties of the virtio-scsi virtual disk
> has changed.  On the working system:
> 
> ==> /sys/block/sda/queue/minimum_io_size <==
> 512
> 
> ==> /sys/block/sda/queue/optimal_io_size <==
> 0
> 
> On the new / broken system:
> 
> ==> /sys/block/sda/queue/minimum_io_size <==
> 33553920
> 
> ==> /sys/block/sda/queue/optimal_io_size <==
> 0
> 
> (all other settings were the same).  I suppose this accounts for it,
> and it could therefore be a problem with qemu rather than LVM.
> 
> Working: qemu-2.11.0-4.fc28.x86_64
> Broken:  qemu-2.12.0-0.5.rc1.fc29.x86_64
> 
> Alasdair Kergon asked me to run the lvm command with -vvvv and I've
> attached those results.


Hi

Could you please retest first with official final release of 4.16 kernel.

There were some 'min()/max()' macro problems - maybe your rc4 build was affected.


Regards


Zdenek

Re: [linux-lvm] "Data alignment must not exceed device size."

[Richard W.M. Jones]

On Tue, Apr 10, 2018 at 04:43:12PM +0200, Zdenek Kabelac wrote:
> Dne 10.4.2018 v 16:00 Richard W.M. Jones napsal(a):
> >On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
> >>Recently in Fedora something changed that stops us from creating small
> >>LVs for testing.
> >>
> >>An example failure with a 64 MB partitioned disk:
> >>
> >># parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
> >>Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
> >># lvm pvcreate --force /dev/sda1
> >>   /dev/sda1: Data alignment must not exceed device size.
> >>   Format-specific initialisation of physical volume /dev/sda1 failed.
> >>   Failed to setup physical volume "/dev/sda1".
> >
> >Interestingly the alignment properties of the virtio-scsi virtual disk
> >has changed.  On the working system:
> >
> >==> /sys/block/sda/queue/minimum_io_size <==
> >512
> >
> >==> /sys/block/sda/queue/optimal_io_size <==
> >0
> >
> >On the new / broken system:
> >
> >==> /sys/block/sda/queue/minimum_io_size <==
> >33553920
> >
> >==> /sys/block/sda/queue/optimal_io_size <==
> >0
> >
> >(all other settings were the same).  I suppose this accounts for it,
> >and it could therefore be a problem with qemu rather than LVM.
> >
> >Working: qemu-2.11.0-4.fc28.x86_64
> >Broken:  qemu-2.12.0-0.5.rc1.fc29.x86_64
> >
> >Alasdair Kergon asked me to run the lvm command with -vvvv and I've
> >attached those results.
> 
> 
> Hi
> 
> Could you please retest first with official final release of 4.16 kernel.

Same thing with 4.16.0-300.fc28.x86_64:

><rescue> cat /sys/block/sda/queue/minimum_io_size
33553920
><rescue> uname -a
Linux (none) 4.16.0-300.fc28.x86_64 #1 SMP Tue Apr 3 03:44:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I believe the faulty min/max macros were added after 4.16.0.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Re: [linux-lvm] "Data alignment must not exceed device size."

[Zdenek Kabelac]

Dne 10.4.2018 v 16:49 Richard W.M. Jones napsal(a):
> On Tue, Apr 10, 2018 at 04:43:12PM +0200, Zdenek Kabelac wrote:
>> Dne 10.4.2018 v 16:00 Richard W.M. Jones napsal(a):
>>> On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
>>>> Recently in Fedora something changed that stops us from creating small
>>>> LVs for testing.
>>>>
>>>> An example failure with a 64 MB partitioned disk:
>>>>
>>>> # parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
>>>> Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
>>>> # lvm pvcreate --force /dev/sda1
>>>>    /dev/sda1: Data alignment must not exceed device size.
>>>>    Format-specific initialisation of physical volume /dev/sda1 failed.
>>>>    Failed to setup physical volume "/dev/sda1".
>>>
>>> Interestingly the alignment properties of the virtio-scsi virtual disk
>>> has changed.  On the working system:
>>>
>
>> Hi
>>
>> Could you please retest first with official final release of 4.16 kernel.
> 
> Same thing with 4.16.0-300.fc28.x86_64:
> 
>> <rescue> cat /sys/block/sda/queue/minimum_io_size
> 33553920


Forget to mention in my previous post - you should be able to override these
bad values reported from kernel with command line 'man pvcreate' options:
--dataalignment & --dataalignmentoffset


Regards

Zdenek

Re: [linux-lvm] "Data alignment must not exceed device size."

[Mike Snitzer]

On Tue, Apr 10 2018 at 10:43am -0400,
Zdenek Kabelac <zkabelac@redhat.com> wrote:

> Dne 10.4.2018 v 16:00 Richard W.M. Jones napsal(a):
> >On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
> >>Recently in Fedora something changed that stops us from creating small
> >>LVs for testing.
> >>
> >>An example failure with a 64 MB partitioned disk:
> >>
> >># parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
> >>Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
> >># lvm pvcreate --force /dev/sda1
> >>   /dev/sda1: Data alignment must not exceed device size.
> >>   Format-specific initialisation of physical volume /dev/sda1 failed.
> >>   Failed to setup physical volume "/dev/sda1".
> >
> >Interestingly the alignment properties of the virtio-scsi virtual disk
> >has changed.  On the working system:
> >
> >==> /sys/block/sda/queue/minimum_io_size <==
> >512
> >
> >==> /sys/block/sda/queue/optimal_io_size <==
> >0
> >
> >On the new / broken system:
> >
> >==> /sys/block/sda/queue/minimum_io_size <==
> >33553920
> >
> >==> /sys/block/sda/queue/optimal_io_size <==
> >0
> >
> >(all other settings were the same).  I suppose this accounts for it,
> >and it could therefore be a problem with qemu rather than LVM.
> >
> >Working: qemu-2.11.0-4.fc28.x86_64
> >Broken:  qemu-2.12.0-0.5.rc1.fc29.x86_64
> >
> >Alasdair Kergon asked me to run the lvm command with -vvvv and I've
> >attached those results.
> 
> 
> Hi
> 
> Could you please retest first with official final release of 4.16 kernel.
> 
> There were some 'min()/max()' macro problems - maybe your rc4 build was affected.

Uptream's min_not_zero() issue was introduced after 4.16 was released
(as part of the 4.17 merge window).

So not related (as Richard already pointed out).

Mike

Re: [linux-lvm] "Data alignment must not exceed device size."

[Zdenek Kabelac]

Dne 10.4.2018 v 16:00 Richard W.M. Jones napsal(a):
> On Tue, Apr 10, 2018 at 09:47:30AM +0100, Richard W.M. Jones wrote:
>> Recently in Fedora something changed that stops us from creating small
>> LVs for testing.
>>
>> An example failure with a 64 MB partitioned disk:
>>
>> # parted -s -- /dev/sda mklabel msdos mkpart primary 128s -128s
>> Warning: The resulting partition is not properly aligned for best performance: 128s % 65535s != 0s
>> # lvm pvcreate --force /dev/sda1
>>    /dev/sda1: Data alignment must not exceed device size.
>>    Format-specific initialisation of physical volume /dev/sda1 failed.
>>    Failed to setup physical volume "/dev/sda1".
> 
> Interestingly the alignment properties of the virtio-scsi virtual disk
> has changed.  On the working system:
> 
> ==> /sys/block/sda/queue/minimum_io_size <==
> 512
> 
> ==> /sys/block/sda/queue/optimal_io_size <==
> 0
> 
> On the new / broken system:
> 
> ==> /sys/block/sda/queue/minimum_io_size <==
> 33553920

What is actually this /dev/sda  as device ?
scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0

Is this some qemu emulated storage ?

This reported value (33553920) doesn't really make sense - and testing over 
loop device doesn't seem to give same result either.

Regards

Zdenek

Re: [linux-lvm] "Data alignment must not exceed device size."

[Richard W.M. Jones]

This turns out to be a bug in qemu 2.12-rc1.

It is fixed by this commit already upstream:

commit 37c51741892a89cf5710f5ac231091fb0a6352c7
Author: Fam Zheng <famz@redhat.com>
Date:   Wed Mar 28 00:41:41 2018 +0800

    scsi-disk: Don't enlarge min_io_size to max_io_size
    
    Some backends report big max_io_sectors. Making min_io_size the same
    value in this case will make it impossible for guest to align memory,
    therefore the disk may not be usable at all.
    
    Do not enlarge them when they are zero.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/