From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx08.extmail.prod.ext.phx2.redhat.com [10.5.110.32]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4A3CD895AC for ; Sat, 2 Feb 2019 23:54:24 +0000 (UTC) Received: from mail-ua1-f70.google.com (mail-ua1-f70.google.com [209.85.222.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5661DC059B8C for ; Sat, 2 Feb 2019 23:54:24 +0000 (UTC) Received: by mail-ua1-f70.google.com with SMTP id d9so867421uaq.20 for ; Sat, 02 Feb 2019 15:54:24 -0800 (PST) MIME-Version: 1.0 From: Nir Soffer Date: Sun, 3 Feb 2019 01:54:10 +0200 Message-ID: Content-Type: multipart/alternative; boundary="0000000000009160610580f1fad3" Subject: [linux-lvm] Mixing devices with different logical or physical block size in oVirt LVM based storage Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: To: linux-lvm@redhat.com Cc: Denis Chaplygin , Mike Snitzer , David Teigland , Vojtech Juranek --0000000000009160610580f1fad3 Content-Type: text/plain; charset="UTF-8" We working on enabling 4k block size in oVirt block storage domain, built using VG on multipath devices on shared storage. We have incomplete support for 4k, added in 2011, for this bug: https://bugzilla.redhat.com/732980 When creating or extending a VG, we check that all PVs are using same logical and phyisical block size, and we store both logical and physical block size in the VG tags. We get the block sizes from /sys/block/dm-X/queue/{logical,physical}_block_size. We also enforce that device physical block size is not smaller than logical block size, This check was added in this patch, trying to enable block size != 512. There is no explanation in the patch or in the review comments why we need to validate this. https://github.com/oVirt/vdsm/commit/7e79153705891a91a06eb31cd642fb209d10ff86 When we start to use a VG, we validate that all the devices are using the stored logical and physical block size. In vdsm itself, we use the logical block size to manage vdsm metadata, assuming that writing and reading one block of logical block size bytes is atomic, and we can read and write different blocks from different hosts at the same time. The relevant code validating PV block sizes is here: https://github.com/oVirt/vdsm/blob/8b043e402f41d8a82b9f832be5f582b8520b38bc/lib/vdsm/storage/lvm.py#L1110 Reading the comments in bug 732980, I don't see anything about physical block size. It looks like this is unnecessary check, and we should check only the logical block size. Regarding mixing devices with different logical block size, according to https://bugzilla.redhat.com/show_bug.cgi?id=732980#c8 We should not extend an LV over devices with different block size, as this will change the device logical block size (e.g change from 512 to 4k), and the change may break the upper layer that already use the device and assume the previous logical block size. Based on this, I think we are ok with limiting VG to devices with same logical block size, so any LV can be extended to any device. I think this code should change to: 1. When creating a VG, check that all PVs use the same logical block size 2. Store the logical block size in the VG tag 3. When extending the VG, check that the new PVs use the same logical block size 4. When starting to use a VG, check that stored logical block size matches PVs logical block size What do you think? Nir --0000000000009160610580f1fad3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
We working on enabling 4k block size in oVirt block storage domain, = built using VG
on multipath devices on shared storage.

We have i= ncomplete support for 4k, added in 2011, for this bug:

=C2=A0 =C2=A0 https://bugzilla.redhat.com/732980

When creating or extending a VG, we check th= at all PVs are using same logical and
phyisical block size, and we store = both logical and physical block size in the VG tags.
We get the block siz= es from /sys/block/dm-X/queue/{logical,physical}_block_size.

We al= so enforce that device physical block size is not smaller than logical bloc= k size,
This check was added in this patch, trying to enable block size != =3D 512. There is no
explanation in the patch or in the review comments w= hy we need to validate this.

=C2=A0 =C2=A0 https://github.com/oVirt/vd= sm/commit/7e79153705891a91a06eb31cd642fb209d10ff86

= When we start to use a VG, we validate that all the devices are using the s= tored logical
and physical block size.

In vdsm itself, we use th= e logical block size to manage vdsm metadata, assuming that writing
and r= eading one block of logical block size bytes is atomic, and we can read and= write
different blocks from different hosts at the same time.

T= he relevant code validating PV block sizes is here:

=C2=A0 =C2=A0 https://github.com/oVirt/vdsm/blob/8b043e402f41d8a= 82b9f832be5f582b8520b38bc/lib/vdsm/storage/lvm.py#L1110
Reading the comments in bug 732980, I don't see anything about phy= sical block size. It looks
like this is unnecessary check, and we should = check only the logical block size.

Regarding mixing devices with = different logical block size, according to

=C2=A0 =C2=A0 https://bugzilla.redhat.com/show_bug.cgi?id=3D7= 32980#c8

We should not extend an LV over devices with different block size, as thi= s will change the device
logical block size (e.g change from 512 to 4k), and the chan= ge may break the upper layer that
= already use the device and assume the previous logi= cal block size.

Based on this, I think we are ok with limiting VG to devices with same log= ical block size, so any=C2=A0
LV can be extended to any device.

I think this cod= e should change to:

1. When creating a VG, check that all PVs use the same l= ogical block size
2. Store the logical bl= ock size in the VG tag
3. When extending = the VG, check that the new PVs use the same logical block size
4. When starting to use a VG, check that stored logic= al block size matches PVs logical block size

What do you think?

Nir

--0000000000009160610580f1fad3--