[linux-lvm] Mixing devices with different logical or physical block size in oVirt LVM based storage

From: Nir Soffer <nsoffer@redhat.com>
To: linux-lvm@redhat.com
Cc: Denis Chaplygin <dchaplyg@redhat.com>,
	Mike Snitzer <msnitzer@redhat.com>,
	David Teigland <teigland@redhat.com>,
	Vojtech Juranek <vjuranek@redhat.com>
Subject: [linux-lvm] Mixing devices with different logical or physical block size in oVirt LVM based storage
Date: Sun, 3 Feb 2019 01:54:10 +0200	[thread overview]
Message-ID: <CAMRbyyv5qcsqmmP0uk+hEBmZJfZ-stV7XWUH23eJDnNMZYs7QA@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 2391 bytes --]

We working on enabling 4k block size in oVirt block storage domain, built
using VG
on multipath devices on shared storage.

We have incomplete support for 4k, added in 2011, for this bug:

    https://bugzilla.redhat.com/732980

When creating or extending a VG, we check that all PVs are using same
logical and
phyisical block size, and we store both logical and physical block size in
the VG tags.
We get the block sizes from
/sys/block/dm-X/queue/{logical,physical}_block_size.

We also enforce that device physical block size is not smaller than logical
block size,
This check was added in this patch, trying to enable block size != 512.
There is no
explanation in the patch or in the review comments why we need to validate
this.

https://github.com/oVirt/vdsm/commit/7e79153705891a91a06eb31cd642fb209d10ff86

When we start to use a VG, we validate that all the devices are using the
stored logical
and physical block size.

In vdsm itself, we use the logical block size to manage vdsm metadata,
assuming that writing
and reading one block of logical block size bytes is atomic, and we can
read and write
different blocks from different hosts at the same time.

The relevant code validating PV block sizes is here:

https://github.com/oVirt/vdsm/blob/8b043e402f41d8a82b9f832be5f582b8520b38bc/lib/vdsm/storage/lvm.py#L1110

Reading the comments in bug 732980, I don't see anything about physical
block size. It looks
like this is unnecessary check, and we should check only the logical block
size.

Regarding mixing devices with different logical block size, according to

    https://bugzilla.redhat.com/show_bug.cgi?id=732980#c8

We should not extend an LV over devices with different block size, as this
will change the device
logical block size (e.g change from 512 to 4k), and the change may break
the upper layer that
already use the device and assume the previous logical block size.

Based on this, I think we are ok with limiting VG to devices with same
logical block size, so any
LV can be extended to any device.

I think this code should change to:

1. When creating a VG, check that all PVs use the same logical block size
2. Store the logical block size in the VG tag
3. When extending the VG, check that the new PVs use the same logical block
size
4. When starting to use a VG, check that stored logical block size matches
PVs logical block size

What do you think?

Nir

[-- Attachment #2: Type: text/html, Size: 6116 bytes --]