* [linux-lvm] lvmcache with vdo - inconsistent block size
@ 2020-09-14 21:44 Gionatan Danti
2020-09-15 18:34 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Gionatan Danti @ 2020-09-14 21:44 UTC (permalink / raw)
To: LVM general discussion and development
Hi all,
I am testing lvmcache with VDO and I have issue with devices block size.
The big & slow VDO device is on top of a 4-disk MD RAID 10 device
(itself on top of dm-integrity). Over the VDO device I created a
thinpool and a thinvol [1]. When adding the cache device to the volume
group via vgextend, I get an error stating "Devices have inconsistent
logical block sizes (4096 and 512)." [2]
Now, I know why the error shows and what i means. However, I don't know
how to force the cache device to act as a 4k sector device, and/if this
is really required to cache a VDO device.
My current workaround is to set VDO with --emulate512=enabled, but this
can be suboptimal and it is not recommended.
Any idea on what I am doing wrong?
Thanks.
[1] Output from lsblk -t:
[root@singularity dev]# lsblk -t | cut -c1-73
NAME ALIGNMENT MIN-IO OPT-IO PHY-SEC
LOG-SEC
sda 0 512 0 512 512
├─sda1 0 512 0 512 512
├─sda2 0 512 0 512 512
│ ├─system-root 0 512 0 512 512
│ └─system-swap 0 512 0 512 512
├─sda3 0 512 0 512 512
└─sda4 0 512 0 512 512
sdb 0 512 0 512 512
└─sdb 0 512 0 512 512
└─md200 0 524288 1048576 512 512
└─vdo1 0 4096 4096 4096 4096
├─vg_kvm-thinpool_tmeta 0 4096 4096 4096 4096
│ └─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
│ ├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
│ └─vg_kvm-thinvol 0 1048576 1048576 4096 4096
└─vg_kvm-thinpool_tdata 0 4096 4096 4096 4096
└─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
└─vg_kvm-thinvol 0 1048576 1048576 4096 4096
sdc 0 512 0 512 512
└─sdc 0 512 0 512 512
└─md200 0 524288 1048576 512 512
└─vdo1 0 4096 4096 4096 4096
├─vg_kvm-thinpool_tmeta 0 4096 4096 4096 4096
│ └─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
│ ├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
│ └─vg_kvm-thinvol 0 1048576 1048576 4096 4096
└─vg_kvm-thinpool_tdata 0 4096 4096 4096 4096
└─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
└─vg_kvm-thinvol 0 1048576 1048576 4096 4096
sdd 0 512 0 512 512
└─sdd 0 512 0 512 512
└─md200 0 524288 1048576 512 512
└─vdo1 0 4096 4096 4096 4096
├─vg_kvm-thinpool_tmeta 0 4096 4096 4096 4096
│ └─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
│ ├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
│ └─vg_kvm-thinvol 0 1048576 1048576 4096 4096
└─vg_kvm-thinpool_tdata 0 4096 4096 4096 4096
└─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
└─vg_kvm-thinvol 0 1048576 1048576 4096 4096
sde 0 512 0 512 512
└─sde 0 512 0 512 512
└─md200 0 524288 1048576 512 512
└─vdo1 0 4096 4096 4096 4096
├─vg_kvm-thinpool_tmeta 0 4096 4096 4096 4096
│ └─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
│ ├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
│ └─vg_kvm-thinvol 0 1048576 1048576 4096 4096
└─vg_kvm-thinpool_tdata 0 4096 4096 4096 4096
└─vg_kvm-thinpool-tpool 0 1048576 1048576 4096 4096
├─vg_kvm-thinpool 0 1048576 1048576 4096 4096
└─vg_kvm-thinvol 0 1048576 1048576 4096 4096
[2] [root@singularity dev]# vgextend vg_kvm /dev/sda4
Devices have inconsistent logical block sizes (4096 and 512).
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [linux-lvm] lvmcache with vdo - inconsistent block size
2020-09-14 21:44 [linux-lvm] lvmcache with vdo - inconsistent block size Gionatan Danti
@ 2020-09-15 18:34 ` Zdenek Kabelac
2020-09-15 22:32 ` Gionatan Danti
0 siblings, 1 reply; 6+ messages in thread
From: Zdenek Kabelac @ 2020-09-15 18:34 UTC (permalink / raw)
To: LVM general discussion and development, Gionatan Danti
Dne 14. 09. 20 v 23:44 Gionatan Danti napsal(a):
> Hi all,
> I am testing lvmcache with VDO and I have issue with devices block size.
>
> The big & slow VDO device is on top of a 4-disk MD RAID 10 device (itself on
> top of dm-integrity). Over the VDO device I created a thinpool and a thinvol
> [1]. When adding the cache device to the volume group via vgextend, I get an
> error stating "Devices have inconsistent logical block sizes (4096 and 512)." [2]
>
> Now, I know why the error shows and what i means. However, I don't know how to
> force the cache device to act as a 4k sector device, and/if this is really
> required to cache a VDO device.
>
> My current workaround is to set VDO with --emulate512=enabled, but this can be
> suboptimal and it is not recommended.
>
> Any idea on what I am doing wrong?
Hi
LVM currently does not support mixing devices of different sector sizes within
a single VG as it brings lot of troubles we have not yet clear vision what
to do with all of them.
Also this combination of provisioned devices is not advised - since you are
combining 2 kind of devices on top of each other and it can be a big problem
to solve recovery case.
On lvm2 side we do not allow to use 'VDO LV' as backend for thin-pool device.
So ATM it's on a user to solve all the possible scenarios that may appear on
such device stack.
Zdenek
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvmcache with vdo - inconsistent block size
2020-09-15 18:34 ` Zdenek Kabelac
@ 2020-09-15 22:32 ` Gionatan Danti
2020-09-17 19:27 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Gionatan Danti @ 2020-09-15 22:32 UTC (permalink / raw)
To: Zdenek Kabelac; +Cc: LVM general discussion and development
Il 2020-09-15 20:34 Zdenek Kabelac ha scritto:
> Dne 14. 09. 20 v 23:44 Gionatan Danti napsal(a):
>> Hi all,
>> I am testing lvmcache with VDO and I have issue with devices block
>> size.
>>
>> The big & slow VDO device is on top of a 4-disk MD RAID 10 device
>> (itself on top of dm-integrity). Over the VDO device I created a
>> thinpool and a thinvol [1]. When adding the cache device to the volume
>> group via vgextend, I get an error stating "Devices have inconsistent
>> logical block sizes (4096 and 512)." [2]
>>
>> Now, I know why the error shows and what i means. However, I don't
>> know how to force the cache device to act as a 4k sector device,
>> and/if this is really required to cache a VDO device.
>>
>> My current workaround is to set VDO with --emulate512=enabled, but
>> this can be suboptimal and it is not recommended.
>>
>> Any idea on what I am doing wrong?
>
> Hi
>
> LVM currently does not support mixing devices of different sector sizes
> within
> a single VG as it brings lot of troubles we have not yet clear vision
> what
> to do with all of them.
Hi Zdenek, yes, I understand. What surprised me is that lvmvdo *can* be
combined with caching, and it does not suffer from this issue. Can you
elaborate on why it works in this case?
> Also this combination of provisioned devices is not advised - since
> you are combining 2 kind of devices on top of each other and it can be
> a big problem
> to solve recovery case.
True.
> On lvm2 side we do not allow to use 'VDO LV' as backend for thin-pool
> device.
I noticed it. However, from what I can read on RedHat docs, thinpool
over VDO device should be perfectly fine (the other way around, not so
much).
> So ATM it's on a user to solve all the possible scenarios that may
> appear on
> such device stack.
>
> Zdenek
Thanks.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvmcache with vdo - inconsistent block size
2020-09-15 22:32 ` Gionatan Danti
@ 2020-09-17 19:27 ` Zdenek Kabelac
2020-09-17 21:41 ` Gionatan Danti
0 siblings, 1 reply; 6+ messages in thread
From: Zdenek Kabelac @ 2020-09-17 19:27 UTC (permalink / raw)
To: LVM general discussion and development, Gionatan Danti
Dne 16. 09. 20 v 0:32 Gionatan Danti napsal(a):
> Il 2020-09-15 20:34 Zdenek Kabelac ha scritto:
>> Dne 14. 09. 20 v 23:44 Gionatan Danti napsal(a):
>>> Hi all,
>>> I am testing lvmcache with VDO and I have issue with devices block size.
>>>
>>> The big & slow VDO device is on top of a 4-disk MD RAID 10 device (itself
>>> on top of dm-integrity). Over the VDO device I created a thinpool and a
>>> thinvol [1]. When adding the cache device to the volume group via vgextend,
>>> I get an error stating "Devices have inconsistent logical block sizes (4096
>>> and 512)." [2]
>>>
>>> Now, I know why the error shows and what i means. However, I don't know how
>>> to force the cache device to act as a 4k sector device, and/if this is
>>> really required to cache a VDO device.
>>>
>>> My current workaround is to set VDO with --emulate512=enabled, but this can
>>> be suboptimal and it is not recommended.
>>>
>>> Any idea on what I am doing wrong?
>>
>> Hi
>>
>> LVM currently does not support mixing devices of different sector sizes within
>> a single VG as it brings lot of troubles we have not yet clear vision what
>> to do with all of them.
>
> Hi Zdenek, yes, I understand. What surprised me is that lvmvdo *can* be
> combined with caching, and it does not suffer from this issue. Can you
> elaborate on why it works in this case?
>
>> Also this combination of provisioned devices is not advised - since
>> you are combining 2 kind of devices on top of each other and it can be
>> a big problem
>> to solve recovery case.
>
> True.
>
>> On lvm2 side we do not allow to use 'VDO LV' as backend for thin-pool device.
>
> I noticed it. However, from what I can read on RedHat docs, thinpool over VDO
> device should be perfectly fine (the other way around, not so much).
>
You've most likely found the bug and this should be likely disable
(and enabled only with some force option).
Problem is, when such device stack is used for XFS - where the 'geometry'
changes, but for some other it's not a big issue (i.e. ext4).
So if you already hit some problem - feel free to open upstream BZ for this issue.
Zdenek
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvmcache with vdo - inconsistent block size
2020-09-17 19:27 ` Zdenek Kabelac
@ 2020-09-17 21:41 ` Gionatan Danti
2020-09-17 21:46 ` Zdenek Kabelac
0 siblings, 1 reply; 6+ messages in thread
From: Gionatan Danti @ 2020-09-17 21:41 UTC (permalink / raw)
To: Zdenek Kabelac; +Cc: LVM general discussion and development
Il 2020-09-17 21:27 Zdenek Kabelac ha scritto:
> You've most likely found the bug and this should be likely disable
> (and enabled only with some force option).
Hi Zdenek, I am not sure about what bug I found - can you be more
explicit?
> Problem is, when such device stack is used for XFS - where the
> 'geometry' changes, but for some other it's not a big issue (i.e.
> ext4).
Again, I am not sure about what we are speaking - generic thinlvm over
VDO, or about the caching issue discussed before? Why this should be an
issue for XFS vs EXT4? From RH docs, I can see VDO + thinlvm + XFS as
being fully supported.
> So if you already hit some problem - feel free to open upstream BZ for
> this issue.
I'll do, but I first need to understand the core issue better.
Thanks.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvmcache with vdo - inconsistent block size
2020-09-17 21:41 ` Gionatan Danti
@ 2020-09-17 21:46 ` Zdenek Kabelac
0 siblings, 0 replies; 6+ messages in thread
From: Zdenek Kabelac @ 2020-09-17 21:46 UTC (permalink / raw)
To: Gionatan Danti; +Cc: LVM general discussion and development
Dne 17. 09. 20 v 23:41 Gionatan Danti napsal(a):
> Il 2020-09-17 21:27 Zdenek Kabelac ha scritto:
>> You've most likely found the bug and this should be likely disable
>> (and enabled only with some force option).
>
> Hi Zdenek, I am not sure about what bug I found - can you be more explicit?
>
>> Problem is, when such device stack is used for XFS - where the
>> 'geometry' changes, but for some other it's not a big issue (i.e.
>> ext4).
XFS doesn't like when 4K device becomes 512b sector based.
> Again, I am not sure about what we are speaking - generic thinlvm over VDO, or
> about the caching issue discussed before? Why this should be an issue for XFS
> vs EXT4? From RH docs, I can see VDO + thinlvm + XFS as being fully supported.
Here you are using separate level of tooling - so you have separate dealing
with errors on each device with its own tooling.
lvm2 is not yet capable to handle this stack on its own.
>
> I'll do, but I first need to understand the core issue better.
Create XFS on 4K sector device - copy it on 512b and use it.
Zdenek
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-17 21:46 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-14 21:44 [linux-lvm] lvmcache with vdo - inconsistent block size Gionatan Danti
2020-09-15 18:34 ` Zdenek Kabelac
2020-09-15 22:32 ` Gionatan Danti
2020-09-17 19:27 ` Zdenek Kabelac
2020-09-17 21:41 ` Gionatan Danti
2020-09-17 21:46 ` Zdenek Kabelac
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.