[PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Luis R. Rodriguez]

Setting up a zoned disks in a generic form is not so trivial. There
is also quite a bit of tribal knowledge with these devices which is not
easy to find.

The currently supplied demo script works but it is not generic enough to be
practical for Linux distributions or even developers which often move
from one kernel to another.

This tries to put a bit of this tribal knowledge into an initial udev
rule for development with the hopes Linux distributions can later
deploy. Three rule are added. One rule is optional for now, it should be
extended later to be more distribution-friendly and then I think this
may be ready for consideration for integration on distributions.

1) scheduler setup
2) backlist f2fs devices
3) run dmsetup for the rest of devices

Note that this udev rule will not work well if you want to use a disk
with f2fs on part of the disk and another filesystem on another part of
the disk. That setup will require manual love so these setups can use
the same backlist on rule 2).

Its not widely known for instance that as of v4.16 it is mandated to use
either deadline or the mq-deadline scheduler for *all* SMR drivers. Its
also been determined that the Linux kernel is not the place to set this up,
so a udev rule *is required* as per latest discussions. This is the
first rule we add.

Furthermore if you are *not* using f2fs you always have to run dmsetup.
dmsetups do not persist, so you currently *always* have to run a custom
sort of script, which is not ideal for Linux distributions. We can invert
this logic into a udev rule to enable users to blacklist disks they know they
want to use f2fs for. This the second optional rule. This blacklisting
can be generalized further in the future with an exception list file, for
instance using INPUT{db} or the like.

The third and final rule added then runs dmsetup for the rest of the disks
using the disk serial number for the new device mapper name.

Note that it is currently easy for users to make a mistake and run mkfs
on the the original disk, not the /dev/mapper/ device for non f2fs
arrangements. If that is done experience shows things can easily fall
apart with alignment *eventually*. We have no generic way today to
error out on this condition and proactively prevent this.

Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
---
 README                    | 10 +++++-
 udev/99-zoned-disks.rules | 78 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 87 insertions(+), 1 deletion(-)
 create mode 100644 udev/99-zoned-disks.rules

diff --git a/README b/README
index 65e96c34fd04..f49541eaabc8 100644
--- a/README
+++ b/README
@@ -168,7 +168,15 @@ Options:
                      reclaiming random zones if the percentage of
                      free random data zones falls below <perc>.
 
-V. Example scripts
+V. Udev zone disk deployment
+============================
+
+A udev rule is provided which enables you to set the IO scheduler, blacklist
+driver to run dmsetup, and runs dmsetup for the rest of the zone drivers.
+If you use this udev rule the below script is not needed. Be sure to mkfs only
+on the resulting /dev/mapper/zone-$serial device you end up with.
+
+VI. Example scripts
 ==================
 
 [[
diff --git a/udev/99-zoned-disks.rules b/udev/99-zoned-disks.rules
new file mode 100644
index 000000000000..e19b738dcc0e
--- /dev/null
+++ b/udev/99-zoned-disks.rules
@@ -0,0 +1,78 @@
+# To use a zone disks first thing you need to:
+#
+# 1) Enable zone disk support in your kernel
+# 2) Use the deadline or mq-deadline scheduler for it - mandated as of v4.16
+# 3) Blacklist devices dedicated for f2fs as of v4.10
+# 4) Run dmsetup other disks
+# 5) Create the filesystem -- NOTE: use mkfs /dev/mapper/zone-serial if
+#    you enabled use dmsetup on the disk.
+# 6) Consider using nofail mount option in case you run an supported kernel
+#
+# You can use this udev rules file for 2) 3) and 4). Further details below.
+#
+# 1) Enable zone disk support in your kernel
+#
+#    o CONFIG_BLK_DEV_ZONED
+#    o CONFIG_DM_ZONED
+#
+# This will let the kernel actually see these devices, ie, via fdisk /dev/sda
+# for instance. Run:
+#
+# 	dmzadm --format /dev/sda
+
+# 2) Set deadline or mq-deadline for all disks which are zoned
+#
+# Zoned disks can only work with the deadline or mq-deadline scheduler. This is
+# mandated for all SMR drives since v4.16. It has been determined this must be
+# done through a udev rule, and the kernel should not set this up for disks.
+# This magic will have to live for *all* zoned disks.
+# XXX: what about distributions that want mq-deadline ? Probably easy for now
+#      to assume deadline and later have a mapping file to enable
+#      mq-deadline for specific serial devices?
+ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
+	ATTR{queue/scheduler}="deadline"
+
+# 3) Blacklist f2fs devices as of v4.10
+# We don't have to run dmsetup on on disks where you want to use f2fs, so you
+# can use this rule to skip dmsetup for it. First get the serial short number.
+#
+#	udevadm info --name=/dev/sda  | grep -i serial_shor
+# XXX: To generalize this for distributions consider using INPUT{db} to or so
+# and then use that to check if the serial number matches one on the database.
+#ACTION=="add", SUBSYSTEM=="block", ENV{ID_SERIAL_SHORT}=="XXA1ZFFF", GOTO="zone_disk_group_end"
+
+# 4) We need to run dmsetup if you want to use other filesystems
+#
+# dmsetup is not persistent, so it needs to be run on upon every boot.  We use
+# the device serial number for the /dev/mapper/ name.
+ACTION=="add", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
+	RUN+="/sbin/dmsetup create zoned-$env{ID_SERIAL_SHORT} --table '0 %s{size} zoned $devnode'", $attr{size}
+
+# 4) Create a filesystem for the device
+#
+# Be 100% sure you use /dev/mapper/zone-$YOUR_DEVICE_SERIAL for the mkfs
+# command as otherwise things can break.
+#
+# XXX: preventing the above proactively in the kernel would be ideal however
+# this may be hard.
+#
+# Once you create the filesystem it will get a UUID.
+#
+# Find out what UUID is, you can do this for instance if your zoned disk is
+# your second device-mapper device, ie dm-1 by:
+#
+# 	ls -l /dev/disk/by-uuid/dm-1
+#
+# To figure out which dm-$number it is, use dmsetup info, the minor number
+# is the $number.
+#
+# 5) Add an etry in /etc/fstab with nofail for example:
+#
+# UUID=99999999-aaaa-bbbb-c1234aaaabbb33456 /media/monster xfs nofail 0 0
+#
+# nofail will ensure system boots fine even if you boot into a kernel which
+# lacks support for the device and so it is not found. Since the UUID will
+# always match the device we don't care if the device moves around the bus
+# on the system. We just need to get the UUID once.
+
+LABEL="zone_disk_group_end"
-- 
2.16.3

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

DQoNCk9uIDYvMTQvMTggMDk6MTEsIEx1aXMgUi4gUm9kcmlndWV6IHdyb3RlOg0KPiBTZXR0aW5n
IHVwIGEgem9uZWQgZGlza3MgaW4gYSBnZW5lcmljIGZvcm0gaXMgbm90IHNvIHRyaXZpYWwuIFRo
ZXJlDQo+IGlzIGFsc28gcXVpdGUgYSBiaXQgb2YgdHJpYmFsIGtub3dsZWRnZSB3aXRoIHRoZXNl
IGRldmljZXMgd2hpY2ggaXMgbm90DQo+IGVhc3kgdG8gZmluZC4NCj4gDQo+IFRoZSBjdXJyZW50
bHkgc3VwcGxpZWQgZGVtbyBzY3JpcHQgd29ya3MgYnV0IGl0IGlzIG5vdCBnZW5lcmljIGVub3Vn
aCB0byBiZQ0KPiBwcmFjdGljYWwgZm9yIExpbnV4IGRpc3RyaWJ1dGlvbnMgb3IgZXZlbiBkZXZl
bG9wZXJzIHdoaWNoIG9mdGVuIG1vdmUNCj4gZnJvbSBvbmUga2VybmVsIHRvIGFub3RoZXIuDQo+
IA0KPiBUaGlzIHRyaWVzIHRvIHB1dCBhIGJpdCBvZiB0aGlzIHRyaWJhbCBrbm93bGVkZ2UgaW50
byBhbiBpbml0aWFsIHVkZXYNCj4gcnVsZSBmb3IgZGV2ZWxvcG1lbnQgd2l0aCB0aGUgaG9wZXMg
TGludXggZGlzdHJpYnV0aW9ucyBjYW4gbGF0ZXINCj4gZGVwbG95LiBUaHJlZSBydWxlIGFyZSBh
ZGRlZC4gT25lIHJ1bGUgaXMgb3B0aW9uYWwgZm9yIG5vdywgaXQgc2hvdWxkIGJlDQo+IGV4dGVu
ZGVkIGxhdGVyIHRvIGJlIG1vcmUgZGlzdHJpYnV0aW9uLWZyaWVuZGx5IGFuZCB0aGVuIEkgdGhp
bmsgdGhpcw0KPiBtYXkgYmUgcmVhZHkgZm9yIGNvbnNpZGVyYXRpb24gZm9yIGludGVncmF0aW9u
IG9uIGRpc3RyaWJ1dGlvbnMuDQo+IA0KPiAxKSBzY2hlZHVsZXIgc2V0dXANCj4gMikgYmFja2xp
c3QgZjJmcyBkZXZpY2VzDQo+IDMpIHJ1biBkbXNldHVwIGZvciB0aGUgcmVzdCBvZiBkZXZpY2Vz
DQo+IA0KPiBOb3RlIHRoYXQgdGhpcyB1ZGV2IHJ1bGUgd2lsbCBub3Qgd29yayB3ZWxsIGlmIHlv
dSB3YW50IHRvIHVzZSBhIGRpc2sNCj4gd2l0aCBmMmZzIG9uIHBhcnQgb2YgdGhlIGRpc2sgYW5k
IGFub3RoZXIgZmlsZXN5c3RlbSBvbiBhbm90aGVyIHBhcnQgb2YNCj4gdGhlIGRpc2suIFRoYXQg
c2V0dXAgd2lsbCByZXF1aXJlIG1hbnVhbCBsb3ZlIHNvIHRoZXNlIHNldHVwcyBjYW4gdXNlDQo+
IHRoZSBzYW1lIGJhY2tsaXN0IG9uIHJ1bGUgMikuDQo+IA0KPiBJdHMgbm90IHdpZGVseSBrbm93
biBmb3IgaW5zdGFuY2UgdGhhdCBhcyBvZiB2NC4xNiBpdCBpcyBtYW5kYXRlZCB0byB1c2UNCj4g
ZWl0aGVyIGRlYWRsaW5lIG9yIHRoZSBtcS1kZWFkbGluZSBzY2hlZHVsZXIgZm9yICphbGwqIFNN
UiBkcml2ZXJzLiBJdHMNCj4gYWxzbyBiZWVuIGRldGVybWluZWQgdGhhdCB0aGUgTGludXgga2Vy
bmVsIGlzIG5vdCB0aGUgcGxhY2UgdG8gc2V0IHRoaXMgdXAsDQo+IHNvIGEgdWRldiBydWxlICpp
cyByZXF1aXJlZCogYXMgcGVyIGxhdGVzdCBkaXNjdXNzaW9ucy4gVGhpcyBpcyB0aGUNCj4gZmly
c3QgcnVsZSB3ZSBhZGQuDQo+IA0KPiBGdXJ0aGVybW9yZSBpZiB5b3UgYXJlICpub3QqIHVzaW5n
IGYyZnMgeW91IGFsd2F5cyBoYXZlIHRvIHJ1biBkbXNldHVwLg0KPiBkbXNldHVwcyBkbyBub3Qg
cGVyc2lzdCwgc28geW91IGN1cnJlbnRseSAqYWx3YXlzKiBoYXZlIHRvIHJ1biBhIGN1c3RvbQ0K
PiBzb3J0IG9mIHNjcmlwdCwgd2hpY2ggaXMgbm90IGlkZWFsIGZvciBMaW51eCBkaXN0cmlidXRp
b25zLiBXZSBjYW4gaW52ZXJ0DQo+IHRoaXMgbG9naWMgaW50byBhIHVkZXYgcnVsZSB0byBlbmFi
bGUgdXNlcnMgdG8gYmxhY2tsaXN0IGRpc2tzIHRoZXkga25vdyB0aGV5DQo+IHdhbnQgdG8gdXNl
IGYyZnMgZm9yLiBUaGlzIHRoZSBzZWNvbmQgb3B0aW9uYWwgcnVsZS4gVGhpcyBibGFja2xpc3Rp
bmcNCj4gY2FuIGJlIGdlbmVyYWxpemVkIGZ1cnRoZXIgaW4gdGhlIGZ1dHVyZSB3aXRoIGFuIGV4
Y2VwdGlvbiBsaXN0IGZpbGUsIGZvcg0KPiBpbnN0YW5jZSB1c2luZyBJTlBVVHtkYn0gb3IgdGhl
IGxpa2UuDQo+IA0KPiBUaGUgdGhpcmQgYW5kIGZpbmFsIHJ1bGUgYWRkZWQgdGhlbiBydW5zIGRt
c2V0dXAgZm9yIHRoZSByZXN0IG9mIHRoZSBkaXNrcw0KPiB1c2luZyB0aGUgZGlzayBzZXJpYWwg
bnVtYmVyIGZvciB0aGUgbmV3IGRldmljZSBtYXBwZXIgbmFtZS4NCj4gDQo+IE5vdGUgdGhhdCBp
dCBpcyBjdXJyZW50bHkgZWFzeSBmb3IgdXNlcnMgdG8gbWFrZSBhIG1pc3Rha2UgYW5kIHJ1biBt
a2ZzDQo+IG9uIHRoZSB0aGUgb3JpZ2luYWwgZGlzaywgbm90IHRoZSAvZGV2L21hcHBlci8gZGV2
aWNlIGZvciBub24gZjJmcw0KPiBhcnJhbmdlbWVudHMuIElmIHRoYXQgaXMgZG9uZSBleHBlcmll
bmNlIHNob3dzIHRoaW5ncyBjYW4gZWFzaWx5IGZhbGwNCj4gYXBhcnQgd2l0aCBhbGlnbm1lbnQg
KmV2ZW50dWFsbHkqLiBXZSBoYXZlIG5vIGdlbmVyaWMgd2F5IHRvZGF5IHRvDQo+IGVycm9yIG91
dCBvbiB0aGlzIGNvbmRpdGlvbiBhbmQgcHJvYWN0aXZlbHkgcHJldmVudCB0aGlzLg0KPiANCj4g
U2lnbmVkLW9mZi1ieTogTHVpcyBSLiBSb2RyaWd1ZXogPG1jZ3JvZkBrZXJuZWwub3JnPg0KPiAt
LS0NCj4gIFJFQURNRSAgICAgICAgICAgICAgICAgICAgfCAxMCArKysrKy0NCj4gIHVkZXYvOTkt
em9uZWQtZGlza3MucnVsZXMgfCA3OCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKw0KPiAgMiBmaWxlcyBjaGFuZ2VkLCA4NyBpbnNlcnRpb25zKCspLCAxIGRl
bGV0aW9uKC0pDQo+ICBjcmVhdGUgbW9kZSAxMDA2NDQgdWRldi85OS16b25lZC1kaXNrcy5ydWxl
cw0KPiANCj4gZGlmZiAtLWdpdCBhL1JFQURNRSBiL1JFQURNRQ0KPiBpbmRleCA2NWU5NmMzNGZk
MDQuLmY0OTU0MWVhYWJjOCAxMDA2NDQNCj4gLS0tIGEvUkVBRE1FDQo+ICsrKyBiL1JFQURNRQ0K
PiBAQCAtMTY4LDcgKzE2OCwxNSBAQCBPcHRpb25zOg0KPiAgICAgICAgICAgICAgICAgICAgICAg
cmVjbGFpbWluZyByYW5kb20gem9uZXMgaWYgdGhlIHBlcmNlbnRhZ2Ugb2YNCj4gICAgICAgICAg
ICAgICAgICAgICAgIGZyZWUgcmFuZG9tIGRhdGEgem9uZXMgZmFsbHMgYmVsb3cgPHBlcmM+Lg0K
PiAgDQo+IC1WLiBFeGFtcGxlIHNjcmlwdHMNCj4gK1YuIFVkZXYgem9uZSBkaXNrIGRlcGxveW1l
bnQNCj4gKz09PT09PT09PT09PT09PT09PT09PT09PT09PT0NCj4gKw0KPiArQSB1ZGV2IHJ1bGUg
aXMgcHJvdmlkZWQgd2hpY2ggZW5hYmxlcyB5b3UgdG8gc2V0IHRoZSBJTyBzY2hlZHVsZXIsIGJs
YWNrbGlzdA0KPiArZHJpdmVyIHRvIHJ1biBkbXNldHVwLCBhbmQgcnVucyBkbXNldHVwIGZvciB0
aGUgcmVzdCBvZiB0aGUgem9uZSBkcml2ZXJzLg0KPiArSWYgeW91IHVzZSB0aGlzIHVkZXYgcnVs
ZSB0aGUgYmVsb3cgc2NyaXB0IGlzIG5vdCBuZWVkZWQuIEJlIHN1cmUgdG8gbWtmcyBvbmx5DQo+
ICtvbiB0aGUgcmVzdWx0aW5nIC9kZXYvbWFwcGVyL3pvbmUtJHNlcmlhbCBkZXZpY2UgeW91IGVu
ZCB1cCB3aXRoLg0KPiArDQo+ICtWSS4gRXhhbXBsZSBzY3JpcHRzDQo+ICA9PT09PT09PT09PT09
PT09PT0NCj4gIA0KPiAgW1sNCj4gZGlmZiAtLWdpdCBhL3VkZXYvOTktem9uZWQtZGlza3MucnVs
ZXMgYi91ZGV2Lzk5LXpvbmVkLWRpc2tzLnJ1bGVzDQo+IG5ldyBmaWxlIG1vZGUgMTAwNjQ0DQo+
IGluZGV4IDAwMDAwMDAwMDAwMC4uZTE5YjczOGRjYzBlDQo+IC0tLSAvZGV2L251bGwNCj4gKysr
IGIvdWRldi85OS16b25lZC1kaXNrcy5ydWxlcw0KPiBAQCAtMCwwICsxLDc4IEBADQo+ICsjIFRv
IHVzZSBhIHpvbmUgZGlza3MgZmlyc3QgdGhpbmcgeW91IG5lZWQgdG86DQo+ICsjDQo+ICsjIDEp
IEVuYWJsZSB6b25lIGRpc2sgc3VwcG9ydCBpbiB5b3VyIGtlcm5lbA0KPiArIyAyKSBVc2UgdGhl
IGRlYWRsaW5lIG9yIG1xLWRlYWRsaW5lIHNjaGVkdWxlciBmb3IgaXQgLSBtYW5kYXRlZCBhcyBv
ZiB2NC4xNg0KPiArIyAzKSBCbGFja2xpc3QgZGV2aWNlcyBkZWRpY2F0ZWQgZm9yIGYyZnMgYXMg
b2YgdjQuMTANCj4gKyMgNCkgUnVuIGRtc2V0dXAgb3RoZXIgZGlza3MNCj4gKyMgNSkgQ3JlYXRl
IHRoZSBmaWxlc3lzdGVtIC0tIE5PVEU6IHVzZSBta2ZzIC9kZXYvbWFwcGVyL3pvbmUtc2VyaWFs
IGlmDQo+ICsjICAgIHlvdSBlbmFibGVkIHVzZSBkbXNldHVwIG9uIHRoZSBkaXNrLg0KPiArIyA2
KSBDb25zaWRlciB1c2luZyBub2ZhaWwgbW91bnQgb3B0aW9uIGluIGNhc2UgeW91IHJ1biBhbiBz
dXBwb3J0ZWQga2VybmVsDQo+ICsjDQo+ICsjIFlvdSBjYW4gdXNlIHRoaXMgdWRldiBydWxlcyBm
aWxlIGZvciAyKSAzKSBhbmQgNCkuIEZ1cnRoZXIgZGV0YWlscyBiZWxvdy4NCj4gKyMNCj4gKyMg
MSkgRW5hYmxlIHpvbmUgZGlzayBzdXBwb3J0IGluIHlvdXIga2VybmVsDQo+ICsjDQo+ICsjICAg
IG8gQ09ORklHX0JMS19ERVZfWk9ORUQNCj4gKyMgICAgbyBDT05GSUdfRE1fWk9ORUQNCj4gKyMN
Cj4gKyMgVGhpcyB3aWxsIGxldCB0aGUga2VybmVsIGFjdHVhbGx5IHNlZSB0aGVzZSBkZXZpY2Vz
LCBpZSwgdmlhIGZkaXNrIC9kZXYvc2RhDQo+ICsjIGZvciBpbnN0YW5jZS4gUnVuOg0KPiArIw0K
PiArIyAJZG16YWRtIC0tZm9ybWF0IC9kZXYvc2RhDQo+ICsNCj4gKyMgMikgU2V0IGRlYWRsaW5l
IG9yIG1xLWRlYWRsaW5lIGZvciBhbGwgZGlza3Mgd2hpY2ggYXJlIHpvbmVkDQo+ICsjDQo+ICsj
IFpvbmVkIGRpc2tzIGNhbiBvbmx5IHdvcmsgd2l0aCB0aGUgZGVhZGxpbmUgb3IgbXEtZGVhZGxp
bmUgc2NoZWR1bGVyLiBUaGlzIGlzDQo+ICsjIG1hbmRhdGVkIGZvciBhbGwgU01SIGRyaXZlcyBz
aW5jZSB2NC4xNi4gSXQgaGFzIGJlZW4gZGV0ZXJtaW5lZCB0aGlzIG11c3QgYmUNCj4gKyMgZG9u
ZSB0aHJvdWdoIGEgdWRldiBydWxlLCBhbmQgdGhlIGtlcm5lbCBzaG91bGQgbm90IHNldCB0aGlz
IHVwIGZvciBkaXNrcy4NCj4gKyMgVGhpcyBtYWdpYyB3aWxsIGhhdmUgdG8gbGl2ZSBmb3IgKmFs
bCogem9uZWQgZGlza3MuDQo+ICsjIFhYWDogd2hhdCBhYm91dCBkaXN0cmlidXRpb25zIHRoYXQg
d2FudCBtcS1kZWFkbGluZSA/IFByb2JhYmx5IGVhc3kgZm9yIG5vdw0KPiArIyAgICAgIHRvIGFz
c3VtZSBkZWFkbGluZSBhbmQgbGF0ZXIgaGF2ZSBhIG1hcHBpbmcgZmlsZSB0byBlbmFibGUNCj4g
KyMgICAgICBtcS1kZWFkbGluZSBmb3Igc3BlY2lmaWMgc2VyaWFsIGRldmljZXM/DQo+ICtBQ1RJ
T049PSJhZGR8Y2hhbmdlIiwgS0VSTkVMPT0ic2QqWyEwLTldIiwgQVRUUlN7cXVldWUvem9uZWR9
PT0iaG9zdC1tYW5hZ2VkIiwgXA0KPiArCUFUVFJ7cXVldWUvc2NoZWR1bGVyfT0iZGVhZGxpbmUi
DQo+ICsNCj4gKyMgMykgQmxhY2tsaXN0IGYyZnMgZGV2aWNlcyBhcyBvZiB2NC4xMA0KPiArIyBX
ZSBkb24ndCBoYXZlIHRvIHJ1biBkbXNldHVwIG9uIG9uIGRpc2tzIHdoZXJlIHlvdSB3YW50IHRv
IHVzZSBmMmZzLCBzbyB5b3UNCj4gKyMgY2FuIHVzZSB0aGlzIHJ1bGUgdG8gc2tpcCBkbXNldHVw
IGZvciBpdC4gRmlyc3QgZ2V0IHRoZSBzZXJpYWwgc2hvcnQgbnVtYmVyLg0KPiArIw0KPiArIwl1
ZGV2YWRtIGluZm8gLS1uYW1lPS9kZXYvc2RhICB8IGdyZXAgLWkgc2VyaWFsX3Nob3INCj4gKyMg
WFhYOiBUbyBnZW5lcmFsaXplIHRoaXMgZm9yIGRpc3RyaWJ1dGlvbnMgY29uc2lkZXIgdXNpbmcg
SU5QVVR7ZGJ9IHRvIG9yIHNvDQo+ICsjIGFuZCB0aGVuIHVzZSB0aGF0IHRvIGNoZWNrIGlmIHRo
ZSBzZXJpYWwgbnVtYmVyIG1hdGNoZXMgb25lIG9uIHRoZSBkYXRhYmFzZS4NCj4gKyNBQ1RJT049
PSJhZGQiLCBTVUJTWVNURU09PSJibG9jayIsIEVOVntJRF9TRVJJQUxfU0hPUlR9PT0iWFhBMVpG
RkYiLCBHT1RPPSJ6b25lX2Rpc2tfZ3JvdXBfZW5kIg0KPiArDQo+ICsjIDQpIFdlIG5lZWQgdG8g
cnVuIGRtc2V0dXAgaWYgeW91IHdhbnQgdG8gdXNlIG90aGVyIGZpbGVzeXN0ZW1zDQo+ICsjDQo+
ICsjIGRtc2V0dXAgaXMgbm90IHBlcnNpc3RlbnQsIHNvIGl0IG5lZWRzIHRvIGJlIHJ1biBvbiB1
cG9uIGV2ZXJ5IGJvb3QuICBXZSB1c2UNCj4gKyMgdGhlIGRldmljZSBzZXJpYWwgbnVtYmVyIGZv
ciB0aGUgL2Rldi9tYXBwZXIvIG5hbWUuDQo+ICtBQ1RJT049PSJhZGQiLCBLRVJORUw9PSJzZCpb
ITAtOV0iLCBBVFRSU3txdWV1ZS96b25lZH09PSJob3N0LW1hbmFnZWQiLCBcDQo+ICsJUlVOKz0i
L3NiaW4vZG1zZXR1cCBjcmVhdGUgem9uZWQtJGVudntJRF9TRVJJQUxfU0hPUlR9IC0tdGFibGUg
JzAgJXN7c2l6ZX0gem9uZWQgJGRldm5vZGUnIiwgJGF0dHJ7c2l6ZX0NCj4gKw0KPiArIyA0KSBD
cmVhdGUgYSBmaWxlc3lzdGVtIGZvciB0aGUgZGV2aWNlDQo+ICsjDQo+ICsjIEJlIDEwMCUgc3Vy
ZSB5b3UgdXNlIC9kZXYvbWFwcGVyL3pvbmUtJFlPVVJfREVWSUNFX1NFUklBTCBmb3IgdGhlIG1r
ZnMNCj4gKyMgY29tbWFuZCBhcyBvdGhlcndpc2UgdGhpbmdzIGNhbiBicmVhay4NCj4gKyMNCj4g
KyMgWFhYOiBwcmV2ZW50aW5nIHRoZSBhYm92ZSBwcm9hY3RpdmVseSBpbiB0aGUga2VybmVsIHdv
dWxkIGJlIGlkZWFsIGhvd2V2ZXINCj4gKyMgdGhpcyBtYXkgYmUgaGFyZC4NCj4gKyMNCj4gKyMg
T25jZSB5b3UgY3JlYXRlIHRoZSBmaWxlc3lzdGVtIGl0IHdpbGwgZ2V0IGEgVVVJRC4NCj4gKyMN
Cj4gKyMgRmluZCBvdXQgd2hhdCBVVUlEIGlzLCB5b3UgY2FuIGRvIHRoaXMgZm9yIGluc3RhbmNl
IGlmIHlvdXIgem9uZWQgZGlzayBpcw0KPiArIyB5b3VyIHNlY29uZCBkZXZpY2UtbWFwcGVyIGRl
dmljZSwgaWUgZG0tMSBieToNCj4gKyMNCj4gKyMgCWxzIC1sIC9kZXYvZGlzay9ieS11dWlkL2Rt
LTENCj4gKyMNCj4gKyMgVG8gZmlndXJlIG91dCB3aGljaCBkbS0kbnVtYmVyIGl0IGlzLCB1c2Ug
ZG1zZXR1cCBpbmZvLCB0aGUgbWlub3IgbnVtYmVyDQo+ICsjIGlzIHRoZSAkbnVtYmVyLg0KPiAr
Iw0KPiArIyA1KSBBZGQgYW4gZXRyeSBpbiAvZXRjL2ZzdGFiIHdpdGggbm9mYWlsIGZvciBleGFt
cGxlOg0KPiArIw0KPiArIyBVVUlEPTk5OTk5OTk5LWFhYWEtYmJiYi1jMTIzNGFhYWFiYmIzMzQ1
NiAvbWVkaWEvbW9uc3RlciB4ZnMgbm9mYWlsIDAgMA0KPiArIw0KPiArIyBub2ZhaWwgd2lsbCBl
bnN1cmUgc3lzdGVtIGJvb3RzIGZpbmUgZXZlbiBpZiB5b3UgYm9vdCBpbnRvIGEga2VybmVsIHdo
aWNoDQo+ICsjIGxhY2tzIHN1cHBvcnQgZm9yIHRoZSBkZXZpY2UgYW5kIHNvIGl0IGlzIG5vdCBm
b3VuZC4gU2luY2UgdGhlIFVVSUQgd2lsbA0KPiArIyBhbHdheXMgbWF0Y2ggdGhlIGRldmljZSB3
ZSBkb24ndCBjYXJlIGlmIHRoZSBkZXZpY2UgbW92ZXMgYXJvdW5kIHRoZSBidXMNCj4gKyMgb24g
dGhlIHN5c3RlbS4gV2UganVzdCBuZWVkIHRvIGdldCB0aGUgVVVJRCBvbmNlLg0KPiArDQo+ICtM
QUJFTD0iem9uZV9kaXNrX2dyb3VwX2VuZCINCg0KQXBwbGllZC4gVGhhbmtzIEx1aXMgIQ0KDQot
LSANCkRhbWllbiBMZSBNb2FsLA0KV2VzdGVybiBEaWdpdGFs

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

On 6/14/18 09:11, Luis R. Rodriguez wrote:
> Setting up a zoned disks in a generic form is not so trivial. There
> is also quite a bit of tribal knowledge with these devices which is not
> easy to find.
> 
> The currently supplied demo script works but it is not generic enough to be
> practical for Linux distributions or even developers which often move
> from one kernel to another.
> 
> This tries to put a bit of this tribal knowledge into an initial udev
> rule for development with the hopes Linux distributions can later
> deploy. Three rule are added. One rule is optional for now, it should be
> extended later to be more distribution-friendly and then I think this
> may be ready for consideration for integration on distributions.
> 
> 1) scheduler setup
> 2) backlist f2fs devices
> 3) run dmsetup for the rest of devices
> 
> Note that this udev rule will not work well if you want to use a disk
> with f2fs on part of the disk and another filesystem on another part of
> the disk. That setup will require manual love so these setups can use
> the same backlist on rule 2).
> 
> Its not widely known for instance that as of v4.16 it is mandated to use
> either deadline or the mq-deadline scheduler for *all* SMR drivers. Its
> also been determined that the Linux kernel is not the place to set this up,
> so a udev rule *is required* as per latest discussions. This is the
> first rule we add.
> 
> Furthermore if you are *not* using f2fs you always have to run dmsetup.
> dmsetups do not persist, so you currently *always* have to run a custom
> sort of script, which is not ideal for Linux distributions. We can invert
> this logic into a udev rule to enable users to blacklist disks they know they
> want to use f2fs for. This the second optional rule. This blacklisting
> can be generalized further in the future with an exception list file, for
> instance using INPUT{db} or the like.
> 
> The third and final rule added then runs dmsetup for the rest of the disks
> using the disk serial number for the new device mapper name.
> 
> Note that it is currently easy for users to make a mistake and run mkfs
> on the the original disk, not the /dev/mapper/ device for non f2fs
> arrangements. If that is done experience shows things can easily fall
> apart with alignment *eventually*. We have no generic way today to
> error out on this condition and proactively prevent this.
> 
> Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
> ---
>  README                    | 10 +++++-
>  udev/99-zoned-disks.rules | 78 +++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 87 insertions(+), 1 deletion(-)
>  create mode 100644 udev/99-zoned-disks.rules
> 
> diff --git a/README b/README
> index 65e96c34fd04..f49541eaabc8 100644
> --- a/README
> +++ b/README
> @@ -168,7 +168,15 @@ Options:
>                       reclaiming random zones if the percentage of
>                       free random data zones falls below <perc>.
>  
> -V. Example scripts
> +V. Udev zone disk deployment
> +============================
> +
> +A udev rule is provided which enables you to set the IO scheduler, blacklist
> +driver to run dmsetup, and runs dmsetup for the rest of the zone drivers.
> +If you use this udev rule the below script is not needed. Be sure to mkfs only
> +on the resulting /dev/mapper/zone-$serial device you end up with.
> +
> +VI. Example scripts
>  ==================
>  
>  [[
> diff --git a/udev/99-zoned-disks.rules b/udev/99-zoned-disks.rules
> new file mode 100644
> index 000000000000..e19b738dcc0e
> --- /dev/null
> +++ b/udev/99-zoned-disks.rules
> @@ -0,0 +1,78 @@
> +# To use a zone disks first thing you need to:
> +#
> +# 1) Enable zone disk support in your kernel
> +# 2) Use the deadline or mq-deadline scheduler for it - mandated as of v4.16
> +# 3) Blacklist devices dedicated for f2fs as of v4.10
> +# 4) Run dmsetup other disks
> +# 5) Create the filesystem -- NOTE: use mkfs /dev/mapper/zone-serial if
> +#    you enabled use dmsetup on the disk.
> +# 6) Consider using nofail mount option in case you run an supported kernel
> +#
> +# You can use this udev rules file for 2) 3) and 4). Further details below.
> +#
> +# 1) Enable zone disk support in your kernel
> +#
> +#    o CONFIG_BLK_DEV_ZONED
> +#    o CONFIG_DM_ZONED
> +#
> +# This will let the kernel actually see these devices, ie, via fdisk /dev/sda
> +# for instance. Run:
> +#
> +# 	dmzadm --format /dev/sda
> +
> +# 2) Set deadline or mq-deadline for all disks which are zoned
> +#
> +# Zoned disks can only work with the deadline or mq-deadline scheduler. This is
> +# mandated for all SMR drives since v4.16. It has been determined this must be
> +# done through a udev rule, and the kernel should not set this up for disks.
> +# This magic will have to live for *all* zoned disks.
> +# XXX: what about distributions that want mq-deadline ? Probably easy for now
> +#      to assume deadline and later have a mapping file to enable
> +#      mq-deadline for specific serial devices?
> +ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
> +	ATTR{queue/scheduler}="deadline"
> +
> +# 3) Blacklist f2fs devices as of v4.10
> +# We don't have to run dmsetup on on disks where you want to use f2fs, so you
> +# can use this rule to skip dmsetup for it. First get the serial short number.
> +#
> +#	udevadm info --name=/dev/sda  | grep -i serial_shor
> +# XXX: To generalize this for distributions consider using INPUT{db} to or so
> +# and then use that to check if the serial number matches one on the database.
> +#ACTION=="add", SUBSYSTEM=="block", ENV{ID_SERIAL_SHORT}=="XXA1ZFFF", GOTO="zone_disk_group_end"
> +
> +# 4) We need to run dmsetup if you want to use other filesystems
> +#
> +# dmsetup is not persistent, so it needs to be run on upon every boot.  We use
> +# the device serial number for the /dev/mapper/ name.
> +ACTION=="add", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
> +	RUN+="/sbin/dmsetup create zoned-$env{ID_SERIAL_SHORT} --table '0 %s{size} zoned $devnode'", $attr{size}
> +
> +# 4) Create a filesystem for the device
> +#
> +# Be 100% sure you use /dev/mapper/zone-$YOUR_DEVICE_SERIAL for the mkfs
> +# command as otherwise things can break.
> +#
> +# XXX: preventing the above proactively in the kernel would be ideal however
> +# this may be hard.
> +#
> +# Once you create the filesystem it will get a UUID.
> +#
> +# Find out what UUID is, you can do this for instance if your zoned disk is
> +# your second device-mapper device, ie dm-1 by:
> +#
> +# 	ls -l /dev/disk/by-uuid/dm-1
> +#
> +# To figure out which dm-$number it is, use dmsetup info, the minor number
> +# is the $number.
> +#
> +# 5) Add an etry in /etc/fstab with nofail for example:
> +#
> +# UUID=99999999-aaaa-bbbb-c1234aaaabbb33456 /media/monster xfs nofail 0 0
> +#
> +# nofail will ensure system boots fine even if you boot into a kernel which
> +# lacks support for the device and so it is not found. Since the UUID will
> +# always match the device we don't care if the device moves around the bus
> +# on the system. We just need to get the UUID once.
> +
> +LABEL="zone_disk_group_end"

Applied. Thanks Luis !

-- 
Damien Le Moal,
Western Digital

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Mike Snitzer]

On Wed, Jun 13 2018 at  8:11pm -0400,
Luis R. Rodriguez <mcgrof@kernel.org> wrote:

> Setting up a zoned disks in a generic form is not so trivial. There
> is also quite a bit of tribal knowledge with these devices which is not
> easy to find.
> 
> The currently supplied demo script works but it is not generic enough to be
> practical for Linux distributions or even developers which often move
> from one kernel to another.
> 
> This tries to put a bit of this tribal knowledge into an initial udev
> rule for development with the hopes Linux distributions can later
> deploy. Three rule are added. One rule is optional for now, it should be
> extended later to be more distribution-friendly and then I think this
> may be ready for consideration for integration on distributions.
> 
> 1) scheduler setup

This is wrong.. if zoned devices are so dependent on deadline or
mq-deadline then the kernel should allow them to be hardcoded.  I know
Jens removed the API to do so but the fact that drivers need to rely on
hacks like this udev rule to get a functional device is proof we need to
allow drivers to impose the scheduler used.

> 2) backlist f2fs devices

There should porbably be support in dm-zoned for detecting whether a
zoned device was formatted with f2fs (assuming there is a known f2fs
superblock)?

> 3) run dmsetup for the rest of devices

automagically running dmsetup directly from udev to create a dm-zoned
target is very much wrong.  It just gets in the way of proper support
that should be add to appropriate tools that admins use to setup their
zoned devices.  For instance, persistent use of dm-zoned target should
be made reliable with a volume manager..

In general this udev script is unwelcome and makes things way worse for
the long-term success of zoned devices.

I don't dispute there is an obvious void for how to properly setup zoned
devices, but this script is _not_ what should fill that void.

So a heartfelt:

Nacked-by: Mike Snitzer <snitzer@redhat.com>

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

T24gVGh1LCAyMDE4LTA2LTE0IGF0IDEwOjAxICswMDAwLCBEYW1pZW4gTGUgTW9hbCB3cm90ZToN
Cj4gQXBwbGllZC4gVGhhbmtzIEx1aXMgIQ0KDQpIZWxsbyBEYW1pZW4sDQoNCkNhbiB0aGlzIHN0
aWxsIGJlIHVuZG9uZT8gSSBhZ3JlZSB3aXRoIE1pa2UgdGhhdCBpdCdzIHdyb25nIHRvIGludm9r
ZQ0KIi9zYmluL2Rtc2V0dXAgY3JlYXRlIC4uLiB6b25lZCAuLi4iIGZyb20gYSB1ZGV2IHJ1bGUu
DQoNClRoYW5rcywNCg0KQmFydC4NCg0KDQoNCg==

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

On Thu, 2018-06-14 at 10:01 +0000, Damien Le Moal wrote:
> Applied. Thanks Luis !

Hello Damien,

Can this still be undone? I agree with Mike that it's wrong to invoke
"/sbin/dmsetup create ... zoned ..." from a udev rule.

Thanks,

Bart.

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Christoph Hellwig]

On Thu, Jun 14, 2018 at 01:39:50PM +0000, Bart Van Assche wrote:
> On Thu, 2018-06-14 at 10:01 +0000, Damien Le Moal wrote:
> > Applied. Thanks Luis !
> 
> Hello Damien,
> 
> Can this still be undone? I agree with Mike that it's wrong to invoke
> "/sbin/dmsetup create ... zoned ..." from a udev rule.

Yes.  We'll really need to verfify the device has dm-zoned metadata
first.  Preferably including a uuid for stable device naming.

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

T24gV2VkLCAyMDE4LTA2LTEzIGF0IDE3OjExIC0wNzAwLCBMdWlzIFIuIFJvZHJpZ3VleiB3cm90
ZToNCj4gVGhpcyB0cmllcyB0byBwdXQgYSBiaXQgb2YgdGhpcyB0cmliYWwga25vd2xlZGdlIGlu
dG8gYW4gaW5pdGlhbCB1ZGV2DQo+IHJ1bGUgZm9yIGRldmVsb3BtZW50IHdpdGggdGhlIGhvcGVz
IExpbnV4IGRpc3RyaWJ1dGlvbnMgY2FuIGxhdGVyDQo+IGRlcGxveS4gVGhyZWUgcnVsZSBhcmUg
YWRkZWQuIE9uZSBydWxlIGlzIG9wdGlvbmFsIGZvciBub3csIGl0IHNob3VsZCBiZQ0KPiBleHRl
bmRlZCBsYXRlciB0byBiZSBtb3JlIGRpc3RyaWJ1dGlvbi1mcmllbmRseSBhbmQgdGhlbiBJIHRo
aW5rIHRoaXMNCj4gbWF5IGJlIHJlYWR5IGZvciBjb25zaWRlcmF0aW9uIGZvciBpbnRlZ3JhdGlv
biBvbiBkaXN0cmlidXRpb25zLg0KPiANCj4gMSkgc2NoZWR1bGVyIHNldHVwDQo+IDIpIGJhY2ts
aXN0IGYyZnMgZGV2aWNlcw0KPiAzKSBydW4gZG1zZXR1cCBmb3IgdGhlIHJlc3Qgb2YgZGV2aWNl
cw0KDQpIZWxsbyBMdWlzLA0KDQpJIHRoaW5rIGl0IGlzIHdyb25nIHRvIHBhY2thZ2UgdGhlIHpv
bmVkIGJsb2NrIGRldmljZSBzY2hlZHVsZXIgcnVsZSBpbiB0aGUNCmRtLXpvbmVkLXRvb2xzIHBh
Y2thZ2UuIFRoYXQgdWRldiBydWxlIHNob3VsZCBiZSBhY3RpdmF0ZWQgd2hldGhlciBvciBub3Qg
dGhlDQpkbS16b25lZC10b29scyBwYWNrYWdlIGhhcyBiZWVuIGluc3RhbGxlZC4gSGF2ZSB5b3Ug
Y29uc2lkZXJlZCB0byBzdWJtaXQgdGhlDQp6b25lZCBibG9jayBkZXZpY2Ugc2NoZWR1bGVyIHJ1
bGUgdG8gdGhlIHN5c3RlbWQgcHJvamVjdCBzaW5jZSB0b2RheSB0aGF0DQpwcm9qZWN0IGluY2x1
ZGVzIGFsbCBiYXNlIHVkZXYgcnVsZXM/DQoNCj4gKyMgWm9uZWQgZGlza3MgY2FuIG9ubHkgd29y
ayB3aXRoIHRoZSBkZWFkbGluZSBvciBtcS1kZWFkbGluZSBzY2hlZHVsZXIuIFRoaXMgaXMNCj4g
KyMgbWFuZGF0ZWQgZm9yIGFsbCBTTVIgZHJpdmVzIHNpbmNlIHY0LjE2LiBJdCBoYXMgYmVlbiBk
ZXRlcm1pbmVkIHRoaXMgbXVzdCBiZQ0KPiArIyBkb25lIHRocm91Z2ggYSB1ZGV2IHJ1bGUsIGFu
ZCB0aGUga2VybmVsIHNob3VsZCBub3Qgc2V0IHRoaXMgdXAgZm9yIGRpc2tzLg0KPiArIyBUaGlz
IG1hZ2ljIHdpbGwgaGF2ZSB0byBsaXZlIGZvciAqYWxsKiB6b25lZCBkaXNrcy4NCj4gKyMgWFhY
OiB3aGF0IGFib3V0IGRpc3RyaWJ1dGlvbnMgdGhhdCB3YW50IG1xLWRlYWRsaW5lID8gUHJvYmFi
bHkgZWFzeSBmb3Igbm93DQo+ICsjICAgICAgdG8gYXNzdW1lIGRlYWRsaW5lIGFuZCBsYXRlciBo
YXZlIGEgbWFwcGluZyBmaWxlIHRvIGVuYWJsZQ0KPiArIyAgICAgIG1xLWRlYWRsaW5lIGZvciBz
cGVjaWZpYyBzZXJpYWwgZGV2aWNlcz8NCj4gK0FDVElPTj09ImFkZHxjaGFuZ2UiLCBLRVJORUw9
PSJzZCpbITAtOV0iLCBBVFRSU3txdWV1ZS96b25lZH09PSJob3N0LW1hbmFnZWQiLCBcDQo+ICsJ
QVRUUntxdWV1ZS9zY2hlZHVsZXJ9PSJkZWFkbGluZSINCg0KSSB0aGluayBpdCBpcyB3cm9uZyB0
byBsaW1pdCB0aGlzIHJ1bGUgdG8gU0NTSSBkaXNrcyBvbmx5LiBXb3JrIGlzIG9uZ29pbmcgdG8N
CmFkZCB6b25lZCBibG9jayBkZXZpY2Ugc3VwcG9ydCB0byB0aGUgbnVsbF9ibGsgZHJpdmVyLiBU
aGF0IGlzIGEgYmxvY2sgZHJpdmVyDQphbmQgbm90IGEgU0NTSSBkcml2ZXIuIEkgdGhpbmsgdGhl
IGFib3ZlIHVkZXYgcnVsZSBzaG91bGQgYXBwbHkgdG8gdGhhdCBibG9jaw0KZHJpdmVyIHRvby4N
Cg0KUmVnYXJkaW5nIGJsay1tcSwgZnJvbSB0aGUgbXEtZGVhZGxpbmUgc291cmNlIGNvZGU6DQoJ
LmVsZXZhdG9yX2FsaWFzID0gImRlYWRsaW5lIiwNCg0KSW4gb3RoZXIgd29yZHMsIHRoZSBuYW1l
ICJkZWFkbGluZSIgc2hvdWxkIHdvcmsgYm90aCBmb3IgbGVnYWN5IGFuZCBmb3IgYmxrLW1xDQpi
bG9jayBkZXZpY2VzLg0KDQpUaGFua3MsDQoNCkJhcnQuDQoNCg0K

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

On Wed, 2018-06-13 at 17:11 -0700, Luis R. Rodriguez wrote:
> This tries to put a bit of this tribal knowledge into an initial udev
> rule for development with the hopes Linux distributions can later
> deploy. Three rule are added. One rule is optional for now, it should be
> extended later to be more distribution-friendly and then I think this
> may be ready for consideration for integration on distributions.
> 
> 1) scheduler setup
> 2) backlist f2fs devices
> 3) run dmsetup for the rest of devices

Hello Luis,

I think it is wrong to package the zoned block device scheduler rule in the
dm-zoned-tools package. That udev rule should be activated whether or not the
dm-zoned-tools package has been installed. Have you considered to submit the
zoned block device scheduler rule to the systemd project since today that
project includes all base udev rules?

> +# Zoned disks can only work with the deadline or mq-deadline scheduler. This is
> +# mandated for all SMR drives since v4.16. It has been determined this must be
> +# done through a udev rule, and the kernel should not set this up for disks.
> +# This magic will have to live for *all* zoned disks.
> +# XXX: what about distributions that want mq-deadline ? Probably easy for now
> +#      to assume deadline and later have a mapping file to enable
> +#      mq-deadline for specific serial devices?
> +ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
> +	ATTR{queue/scheduler}="deadline"

I think it is wrong to limit this rule to SCSI disks only. Work is ongoing to
add zoned block device support to the null_blk driver. That is a block driver
and not a SCSI driver. I think the above udev rule should apply to that block
driver too.

Regarding blk-mq, from the mq-deadline source code:
	.elevator_alias = "deadline",

In other words, the name "deadline" should work both for legacy and for blk-mq
block devices.

Thanks,

Bart.

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

T24gVGh1LCAyMDE4LTA2LTE0IGF0IDA4OjM4IC0wNDAwLCBNaWtlIFNuaXR6ZXIgd3JvdGU6DQo+
IE9uIFdlZCwgSnVuIDEzIDIwMTggYXQgIDg6MTFwbSAtMDQwMCwNCj4gTHVpcyBSLiBSb2RyaWd1
ZXogPG1jZ3JvZkBrZXJuZWwub3JnPiB3cm90ZToNCj4gPiAxKSBzY2hlZHVsZXIgc2V0dXANCj4g
DQo+IFRoaXMgaXMgd3JvbmcuLiBpZiB6b25lZCBkZXZpY2VzIGFyZSBzbyBkZXBlbmRlbnQgb24g
ZGVhZGxpbmUgb3INCj4gbXEtZGVhZGxpbmUgdGhlbiB0aGUga2VybmVsIHNob3VsZCBhbGxvdyB0
aGVtIHRvIGJlIGhhcmRjb2RlZC4gIEkga25vdw0KPiBKZW5zIHJlbW92ZWQgdGhlIEFQSSB0byBk
byBzbyBidXQgdGhlIGZhY3QgdGhhdCBkcml2ZXJzIG5lZWQgdG8gcmVseSBvbg0KPiBoYWNrcyBs
aWtlIHRoaXMgdWRldiBydWxlIHRvIGdldCBhIGZ1bmN0aW9uYWwgZGV2aWNlIGlzIHByb29mIHdl
IG5lZWQgdG8NCj4gYWxsb3cgZHJpdmVycyB0byBpbXBvc2UgdGhlIHNjaGVkdWxlciB1c2VkLg0K
DQpIZWxsbyBNaWtlLA0KDQpBcyB5b3Uga25vdyB0aGUgTGludXgga2VybmVsIGJsb2NrIGxheWVy
IHN0YWNrIGNhbiByZW9yZGVyIHJlcXVlc3RzLiBIb3dldmVyLA0KZm9yIHpvbmVkIGJsb2NrIGRl
dmljZXMgaXQgaXMgZXNzZW50aWFsIHRoYXQgdGhlIGJsb2NrIGRldmljZSByZWNlaXZlcyB3cml0
ZQ0KcmVxdWVzdHMgaW4gdGhlIHNhbWUgb3JkZXIgYXMgdGhlc2Ugd2VyZSBzdWJtaXR0ZWQgYnkg
dGhlICh1c2VyIHNwYWNlKQ0KYXBwbGljYXRpb24gb3IgYnkgdGhlIChrZXJuZWwpIGZpbGVzeXN0
ZW0uIEFmdGVyIGEgbG9uZyBkZWJhdGUgdGhlIGNob2ljZSB3YXMNCm1hZGUgdG8gbWFrZSBJL08g
c2NoZWR1bGVycyByZXNwb25zaWJsZSBmb3IgZ3VhcmFudGVlaW5nIHRvIHByZXNlcnZlIHRoZSB3
cml0ZQ0Kb3JkZXIuIFRvZGF5IG9ubHkgdGhlIGRlYWRsaW5lIHNjaGVkdWxlciBndWFyYW50ZWVz
IHRoYXQgdGhlIHdyaXRlIG9yZGVyIGlzDQpwcmVzZXJ2ZWQuIEhlbmNlIHRoZSB1ZGV2IHJ1bGUg
dGhhdCBzZXRzIHRoZSBkZWFkbGluZSBzY2hlZHVsZXIgZm9yIHpvbmVkDQpibG9jayBkZXZpY2Vz
Lg0KDQpCYXJ0Lg0KDQoNCg0KDQo=

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Bart Van Assche]

On Thu, 2018-06-14 at 08:38 -0400, Mike Snitzer wrote:
> On Wed, Jun 13 2018 at  8:11pm -0400,
> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> > 1) scheduler setup
> 
> This is wrong.. if zoned devices are so dependent on deadline or
> mq-deadline then the kernel should allow them to be hardcoded.  I know
> Jens removed the API to do so but the fact that drivers need to rely on
> hacks like this udev rule to get a functional device is proof we need to
> allow drivers to impose the scheduler used.

Hello Mike,

As you know the Linux kernel block layer stack can reorder requests. However,
for zoned block devices it is essential that the block device receives write
requests in the same order as these were submitted by the (user space)
application or by the (kernel) filesystem. After a long debate the choice was
made to make I/O schedulers responsible for guaranteeing to preserve the write
order. Today only the deadline scheduler guarantees that the write order is
preserved. Hence the udev rule that sets the deadline scheduler for zoned
block devices.

Bart.

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Luis R. Rodriguez]

On Thu, Jun 14, 2018 at 08:38:06AM -0400, Mike Snitzer wrote:
> On Wed, Jun 13 2018 at  8:11pm -0400,
> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> 
> > Setting up a zoned disks in a generic form is not so trivial. There
> > is also quite a bit of tribal knowledge with these devices which is not
> > easy to find.
> > 
> > The currently supplied demo script works but it is not generic enough to be
> > practical for Linux distributions or even developers which often move
> > from one kernel to another.
> > 
> > This tries to put a bit of this tribal knowledge into an initial udev
> > rule for development with the hopes Linux distributions can later
> > deploy. Three rule are added. One rule is optional for now, it should be
> > extended later to be more distribution-friendly and then I think this
> > may be ready for consideration for integration on distributions.
> > 
> > 1) scheduler setup
> 
> This is wrong.. if zoned devices are so dependent on deadline or
> mq-deadline then the kernel should allow them to be hardcoded.  I know
> Jens removed the API to do so but the fact that drivers need to rely on
> hacks like this udev rule to get a functional device is proof we need to
> allow drivers to impose the scheduler used.

This is the point to the patch as well, I actually tend to agree with you,
and I had tried to draw up a patch to do just that, however its *not* possible
today to do this and would require some consensus. So from what I can tell
we *have* to live with this one or a form of it. Ie a file describing which
disk serial gets deadline and which one gets mq-deadline.

Jens?

Anyway, let's assume this is done in the kernel, which one would use deadline,
which one would use mq-deadline?

> > 2) backlist f2fs devices
> 
> There should porbably be support in dm-zoned for detecting whether a
> zoned device was formatted with f2fs (assuming there is a known f2fs
> superblock)?

Not sure what you mean. Are you suggesting we always setup dm-zoned for
all zoned disks and just make an excemption on dm-zone code to somehow
use the disk directly if a filesystem supports zoned disks directly somehow?

f2fs does not require dm-zoned. What would be required is a bit more complex
given one could dedicate portions of the disk to f2fs and other portions to
another filesystem, which would require dm-zoned.

Also filesystems which *do not* support zoned disks should *not* be allowing
direct setup. Today that's all filesystems other than f2fs, in the future
that may change. Those are bullets we are allowing to trigger for users
just waiting to shot themselves on the foot with.

So who's going to work on all the above?

The point of the udev script is to illustrate the pains to properly deploy
zoned disks on distributions today and without a roadmap... this is what
at least I need on my systems today to reasonably deploy these disks for
my own development.

Consensus is indeed needed for a broader picture.

> > 3) run dmsetup for the rest of devices
> 
> automagically running dmsetup directly from udev to create a dm-zoned
> target is very much wrong.  It just gets in the way of proper support
> that should be add to appropriate tools that admins use to setup their
> zoned devices.  For instance, persistent use of dm-zoned target should
> be made reliable with a volume manager..

Ah yes, but who's working on that? How long will it take?

I agree it is odd to expect one to use dmsetup and then use a volume manager on
top of it, if we can just add proper support onto the volume manager... then
that's a reasonable way to go.

But *we're not there* yet, and as-is today, what is described in the udev
script is the best we can do for a generic setup.

> In general this udev script is unwelcome and makes things way worse for
> the long-term success of zoned devices.

dm-zoned-tools does not acknowledge in any way a roadmap, and just provides
a script, which IMHO is less generic and less distribution friendly. Having
a udev rule in place to demonstrate the current state of affairs IMHO is
more scalable demonstrates the issues better than the script.

If we have an agreed upon long term strategy lets document that. But from
what I gather we are not even in consensus with regards to the scheduler
stuff. If we have consensus on the other stuff lets document that as
dm-zoned-tools is the only place I think folks could find to reasonably
deploy these things.

> I don't dispute there is an obvious void for how to properly setup zoned
> devices, but this script is _not_ what should fill that void.

Good to know! Again, consider it as an alternative to the script.

I'm happy to adapt the language and supply it only as an example script
developers can use, but we can't leave users hanging as well. Let's at
least come up with a plan which we seem to agree on and document that.

  Luis

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Luis R. Rodriguez]

On Thu, Jun 14, 2018 at 04:19:23PM +0000, Bart Van Assche wrote:
> On Wed, 2018-06-13 at 17:11 -0700, Luis R. Rodriguez wrote:
> > This tries to put a bit of this tribal knowledge into an initial udev
> > rule for development with the hopes Linux distributions can later
> > deploy. Three rule are added. One rule is optional for now, it should be
> > extended later to be more distribution-friendly and then I think this
> > may be ready for consideration for integration on distributions.
> > 
> > 1) scheduler setup
> > 2) backlist f2fs devices
> > 3) run dmsetup for the rest of devices
> 
> Hello Luis,
> 
> I think it is wrong to package the zoned block device scheduler rule in the
> dm-zoned-tools package. That udev rule should be activated whether or not the
> dm-zoned-tools package has been installed. Have you considered to submit the
> zoned block device scheduler rule to the systemd project since today that
> project includes all base udev rules?

Nope, this is a udev rule intended for developers wishing to brush up on our
state of affairs. All I wanted to do is to get my own drive to work at home in
a reasonably reliable and generic form, which also allowed me to hop around
kernels without a fatal issue.

Clearly there is much to discuss still and a roadmap to illustrate. We should
update the documentation to reflect this first, and based on that enable then
distributions based on that discussion. We should have short term and long term
plans. That discussion may have taken place already so forgive me for not
knowing about it, I did try to read as much from the code and existing tools
and just didn't find anything else to be clear on it.

> > +# Zoned disks can only work with the deadline or mq-deadline scheduler. This is
> > +# mandated for all SMR drives since v4.16. It has been determined this must be
> > +# done through a udev rule, and the kernel should not set this up for disks.
> > +# This magic will have to live for *all* zoned disks.
> > +# XXX: what about distributions that want mq-deadline ? Probably easy for now
> > +#      to assume deadline and later have a mapping file to enable
> > +#      mq-deadline for specific serial devices?
> > +ACTION=="add|change", KERNEL=="sd*[!0-9]", ATTRS{queue/zoned}=="host-managed", \
> > +	ATTR{queue/scheduler}="deadline"
> 
> I think it is wrong to limit this rule to SCSI disks only. Work is ongoing to
> add zoned block device support to the null_blk driver. That is a block driver
> and not a SCSI driver. I think the above udev rule should apply to that block
> driver too.

Sure patches welcomed :)

> Regarding blk-mq, from the mq-deadline source code:
> 	.elevator_alias = "deadline",
> 
> In other words, the name "deadline" should work both for legacy and for blk-mq
> block devices.

Groovy that helps.

  Luis

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Luis R. Rodriguez]

On Thu, Jun 14, 2018 at 07:37:19PM +0200, Luis R. Rodriguez wrote:
> 
> Ie a file describing which
> disk serial gets deadline and which one gets mq-deadline.
> Anyway, let's assume this is done in the kernel, which one would use deadline,
> which one would use mq-deadline?

Nevermind this, deadline will work, Bart already stated why.

 Luis

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Mike Snitzer]

On Thu, Jun 14 2018 at  1:37pm -0400,
Luis R. Rodriguez <mcgrof@kernel.org> wrote:

> On Thu, Jun 14, 2018 at 08:38:06AM -0400, Mike Snitzer wrote:
> > On Wed, Jun 13 2018 at  8:11pm -0400,
> > Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> > 
> > > Setting up a zoned disks in a generic form is not so trivial. There
> > > is also quite a bit of tribal knowledge with these devices which is not
> > > easy to find.
> > > 
> > > The currently supplied demo script works but it is not generic enough to be
> > > practical for Linux distributions or even developers which often move
> > > from one kernel to another.
> > > 
> > > This tries to put a bit of this tribal knowledge into an initial udev
> > > rule for development with the hopes Linux distributions can later
> > > deploy. Three rule are added. One rule is optional for now, it should be
> > > extended later to be more distribution-friendly and then I think this
> > > may be ready for consideration for integration on distributions.
> > > 
> > > 1) scheduler setup
> > 
> > This is wrong.. if zoned devices are so dependent on deadline or
> > mq-deadline then the kernel should allow them to be hardcoded.  I know
> > Jens removed the API to do so but the fact that drivers need to rely on
> > hacks like this udev rule to get a functional device is proof we need to
> > allow drivers to impose the scheduler used.
> 
> This is the point to the patch as well, I actually tend to agree with you,
> and I had tried to draw up a patch to do just that, however its *not* possible
> today to do this and would require some consensus. So from what I can tell
> we *have* to live with this one or a form of it. Ie a file describing which
> disk serial gets deadline and which one gets mq-deadline.
> 
> Jens?
> 
> Anyway, let's assume this is done in the kernel, which one would use deadline,
> which one would use mq-deadline?

The zoned storage driver needs to make that call based on what mode it
is in.  If it is using blk-mq then it selects mq-deadline, otherwise
deadline.
 
> > > 2) backlist f2fs devices
> > 
> > There should porbably be support in dm-zoned for detecting whether a
> > zoned device was formatted with f2fs (assuming there is a known f2fs
> > superblock)?
> 
> Not sure what you mean. Are you suggesting we always setup dm-zoned for
> all zoned disks and just make an excemption on dm-zone code to somehow
> use the disk directly if a filesystem supports zoned disks directly somehow?

No, I'm saying that a udev rule wouldn't be needed if dm-zoned just
errored out if asked to consume disks that already have an f2fs
superblock.  And existing filesystems should get conflicting superblock
awareness "for free" if blkid or whatever is trained to be aware of
f2fs's superblock.
 
> f2fs does not require dm-zoned. What would be required is a bit more complex
> given one could dedicate portions of the disk to f2fs and other portions to
> another filesystem, which would require dm-zoned.
> 
> Also filesystems which *do not* support zoned disks should *not* be allowing
> direct setup. Today that's all filesystems other than f2fs, in the future
> that may change. Those are bullets we are allowing to trigger for users
> just waiting to shot themselves on the foot with.
> 
> So who's going to work on all the above?

It should take care of itself if existing tools are trained to be aware
of new signatures.  E.g. ext4 and xfs already are aware of one another
so that you cannot reformat a device with the other unless force is
given.

Same kind of mutual exclussion needs to happen for zoned devices.

So the zoned device tools, dm-zoned, f2fs, whatever.. they need to be
updated to not step on each others toes.  And other filesystems' tools
need to be updated to be zoned device aware.

> The point of the udev script is to illustrate the pains to properly deploy
> zoned disks on distributions today and without a roadmap... this is what
> at least I need on my systems today to reasonably deploy these disks for
> my own development.
> 
> Consensus is indeed needed for a broader picture.

Yeap.

> > > 3) run dmsetup for the rest of devices
> > 
> > automagically running dmsetup directly from udev to create a dm-zoned
> > target is very much wrong.  It just gets in the way of proper support
> > that should be add to appropriate tools that admins use to setup their
> > zoned devices.  For instance, persistent use of dm-zoned target should
> > be made reliable with a volume manager..
> 
> Ah yes, but who's working on that? How long will it take?

No idea, as is (from my vantage point) there is close to zero demand for
zoned devices.  It won't be a priority until enough customers are asking
for it.

> I agree it is odd to expect one to use dmsetup and then use a volume manager on
> top of it, if we can just add proper support onto the volume manager... then
> that's a reasonable way to go.
> 
> But *we're not there* yet, and as-is today, what is described in the udev
> script is the best we can do for a generic setup.

Just because doing things right takes work doesn't mean it makes sense
to elevate this udev script to be packaged in some upstream project like
udev or whatever.

But if SUSE or some other distro wants to ship it that is fine.

> > In general this udev script is unwelcome and makes things way worse for
> > the long-term success of zoned devices.
> 
> dm-zoned-tools does not acknowledge in any way a roadmap, and just provides
> a script, which IMHO is less generic and less distribution friendly. Having
> a udev rule in place to demonstrate the current state of affairs IMHO is
> more scalable demonstrates the issues better than the script.
> 
> If we have an agreed upon long term strategy lets document that. But from
> what I gather we are not even in consensus with regards to the scheduler
> stuff. If we have consensus on the other stuff lets document that as
> dm-zoned-tools is the only place I think folks could find to reasonably
> deploy these things.

I'm sure Damien and others will have something to say here.

> > I don't dispute there is an obvious void for how to properly setup zoned
> > devices, but this script is _not_ what should fill that void.
> 
> Good to know! Again, consider it as an alternative to the script.
> 
> I'm happy to adapt the language and supply it only as an example script
> developers can use, but we can't leave users hanging as well. Let's at
> least come up with a plan which we seem to agree on and document that.

Best to try to get Damien and others more invested in zoned devices to
help you take up your cause.  I think it is worthwhile to develop a
strategy.  But it needs to be done in terms of the norms of the existing
infrastructure we all make use of today.  So first step is making
existing tools zoned device aware (even if to reject such devices).

Mike

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

TWlrZSwNCg0KT24gNi8xNC8xOCAyMTozOCwgTWlrZSBTbml0emVyIHdyb3RlOg0KPiBPbiBXZWQs
IEp1biAxMyAyMDE4IGF0ICA4OjExcG0gLTA0MDAsDQo+IEx1aXMgUi4gUm9kcmlndWV6IDxtY2dy
b2ZAa2VybmVsLm9yZz4gd3JvdGU6DQo+IA0KPj4gU2V0dGluZyB1cCBhIHpvbmVkIGRpc2tzIGlu
IGEgZ2VuZXJpYyBmb3JtIGlzIG5vdCBzbyB0cml2aWFsLiBUaGVyZQ0KPj4gaXMgYWxzbyBxdWl0
ZSBhIGJpdCBvZiB0cmliYWwga25vd2xlZGdlIHdpdGggdGhlc2UgZGV2aWNlcyB3aGljaCBpcyBu
b3QNCj4+IGVhc3kgdG8gZmluZC4NCj4+DQo+PiBUaGUgY3VycmVudGx5IHN1cHBsaWVkIGRlbW8g
c2NyaXB0IHdvcmtzIGJ1dCBpdCBpcyBub3QgZ2VuZXJpYyBlbm91Z2ggdG8gYmUNCj4+IHByYWN0
aWNhbCBmb3IgTGludXggZGlzdHJpYnV0aW9ucyBvciBldmVuIGRldmVsb3BlcnMgd2hpY2ggb2Z0
ZW4gbW92ZQ0KPj4gZnJvbSBvbmUga2VybmVsIHRvIGFub3RoZXIuDQo+Pg0KPj4gVGhpcyB0cmll
cyB0byBwdXQgYSBiaXQgb2YgdGhpcyB0cmliYWwga25vd2xlZGdlIGludG8gYW4gaW5pdGlhbCB1
ZGV2DQo+PiBydWxlIGZvciBkZXZlbG9wbWVudCB3aXRoIHRoZSBob3BlcyBMaW51eCBkaXN0cmli
dXRpb25zIGNhbiBsYXRlcg0KPj4gZGVwbG95LiBUaHJlZSBydWxlIGFyZSBhZGRlZC4gT25lIHJ1
bGUgaXMgb3B0aW9uYWwgZm9yIG5vdywgaXQgc2hvdWxkIGJlDQo+PiBleHRlbmRlZCBsYXRlciB0
byBiZSBtb3JlIGRpc3RyaWJ1dGlvbi1mcmllbmRseSBhbmQgdGhlbiBJIHRoaW5rIHRoaXMNCj4+
IG1heSBiZSByZWFkeSBmb3IgY29uc2lkZXJhdGlvbiBmb3IgaW50ZWdyYXRpb24gb24gZGlzdHJp
YnV0aW9ucy4NCj4+DQo+PiAxKSBzY2hlZHVsZXIgc2V0dXANCj4gDQo+IFRoaXMgaXMgd3Jvbmcu
LiBpZiB6b25lZCBkZXZpY2VzIGFyZSBzbyBkZXBlbmRlbnQgb24gZGVhZGxpbmUgb3INCj4gbXEt
ZGVhZGxpbmUgdGhlbiB0aGUga2VybmVsIHNob3VsZCBhbGxvdyB0aGVtIHRvIGJlIGhhcmRjb2Rl
ZC4gIEkga25vdw0KPiBKZW5zIHJlbW92ZWQgdGhlIEFQSSB0byBkbyBzbyBidXQgdGhlIGZhY3Qg
dGhhdCBkcml2ZXJzIG5lZWQgdG8gcmVseSBvbg0KPiBoYWNrcyBsaWtlIHRoaXMgdWRldiBydWxl
IHRvIGdldCBhIGZ1bmN0aW9uYWwgZGV2aWNlIGlzIHByb29mIHdlIG5lZWQgdG8NCj4gYWxsb3cg
ZHJpdmVycyB0byBpbXBvc2UgdGhlIHNjaGVkdWxlciB1c2VkLg0KDQpJIGFncmVlLiBTd2l0Y2hp
bmcgc2NoZWR1bGVyIGluIHRoZSBrZXJuZWwgZHVyaW5nIGRldmljZSBwcm9iZS9icmluZy11cA0K
d291bGQgYmUgbXkgcHJlZmVycmVkIGNob2ljZS4gQnV0IHVudGlsIHdlIGNvbWUgdG8gYSBjb25z
ZW5zdXMgb24gdGhlDQpiZXN0IHdheSB0byBkbyB0aGlzLCBJIHRoaW5rIHRoYXQgdGhpcyB1ZGV2
IHJ1bGUgaXMgdXNlZnVsIHNpbmNlIHRoZQ0KInNjaGVkdWxlcj0iIGtlcm5lbCBwYXJhbWV0ZXIg
aXMgcmF0aGVyIGhlYXZ5IGhhbmRlZCBhbmQgYXBwbGllcyB0byBhbGwNCnNpbmdsZSBxdWV1ZSBk
ZXZpY2VzLiBOb3QgdG8gbWVudGlvbiB0aGF0IGFkZGluZyB0aGlzIHBhcmFtZXRlciB0byB0aGUN
Cmtlcm5lbCBhcmd1bWVudHMgaXMgaW4gZXNzZW5jZSBzaW1pbGFyIHRvIHRoZSB1ZGV2IHJ1bGUg
YWRkaXRpb246IGFjdGlvbg0KZnJvbSB0aGUgc3lzdGVtIHVzZXIgaXMgbmVjZXNzYXJ5IHRvIGFj
aGlldmUgYSBjb3JyZWN0IGNvbmZpZ3VyYXRpb24NCmV2ZW4gdGhvdWdoIHRoYXQgY291bGQgZWFz
aWx5IGJlIGRvbmUgYXV0b21hdGljYWxseSBmcm9tIHdpdGhpbiB0aGUNCmJsb2NrIGxheWVyLg0K
DQpJbiB0aGUgbWVhbnRpbWUsIGRvY3VtZW50aW5nIHByb3Blcmx5IHRoYXQgdGhlIGRlYWRsaW5l
IHNjaGVkdWxlciBoYXMgYQ0Kc3BlY2lhbCByZWxhdGlvbnNoaXAgd2l0aCB6b25lZCBibG9jayBk
ZXZpY2VzIGlzIHN0aWxsIGEgbmljZSB0aGluZyB0bw0KZG8uIEJ1dCB5ZXMsIGRtLXpvbmVkLXRv
b2xzIG1heSBub3QgYmUgdGhlIGJlc3QgcGxhY2UgdG8gZG8gdGhhdC4gQQ0Kc2ltcGxlIHRleHQg
ZmlsZSB1bmRlciBEb2N1bWVudGF0aW9uL2Jsb2NrIG1heSBiZSBiZXR0ZXIuIEF0IHRoZSB2ZXJ5
DQpsZWFzdCwgRG9jdW1lbnRhdGlvbi9ibG9jay9kZWFkbGluZS1pb3NjaGVkLnR4dCBzaG91bGQg
aGF2ZSBzb21lIG1lbnRpb24NCm9mIGl0cyBhbG1vc3QgbWFuZGF0b3J5IHVzZSB3aXRoIHpvbmVk
IGJsb2NrIGRldmljZXMuIEkgd2lsbCB3b3JrIG9uIHRoYXQuDQoNCj4+IDIpIGJhY2tsaXN0IGYy
ZnMgZGV2aWNlcw0KPiANCj4gVGhlcmUgc2hvdWxkIHBvcmJhYmx5IGJlIHN1cHBvcnQgaW4gZG0t
em9uZWQgZm9yIGRldGVjdGluZyB3aGV0aGVyIGENCj4gem9uZWQgZGV2aWNlIHdhcyBmb3JtYXR0
ZWQgd2l0aCBmMmZzIChhc3N1bWluZyB0aGVyZSBpcyBhIGtub3duIGYyZnMNCj4gc3VwZXJibG9j
ayk/DQoNClRoYXQgd291bGQgY2VydGFpbmx5IGJlIG5pY2UgdG8gaGF2ZSBhbmQgd291bGQgbm90
IGJlIHRvbyBoYXJkIHRvIGNvZGUNCmluIGRtemFkbS4gSSBjYW4gYWRkIHRoYXQgYW5kIGRvIHdo
YXQgZm9yIGluc3RhbmNlIG1rZnMuZXh0NCBvciBta2ZzLnhmcw0KZG8sIHdoaWNoIGlzIGFzayBm
b3IgY29uZmlybWF0aW9uIG9yIGJhaWwgb3V0IGlmIGFuIGV4aXN0aW5nIHZhbGlkIEZTDQpmb3Jt
YXQgaXMgZGV0ZWN0ZWQgb24gdGhlIGRpc2suDQoNClRoYXQgc2FpZCwgc3VjaCB0ZXN0IGFyZSBm
YXIgZnJvbSBjb21tb24uIG1kYWRtIGZvciBpbnN0YW5jZSB3aWxsDQpoYXBwaWx5IGZvcm1hdCBh
bmQgc3RhcnQgYW4gYXJyYXkgdXNpbmcgdW5tb3VudGVkIGRpc2tzIHdpdGggdmFsaWQgZmlsZQ0K
c3lzdGVtcyBvbiB0aGVtLg0KDQo+PiAzKSBydW4gZG1zZXR1cCBmb3IgdGhlIHJlc3Qgb2YgZGV2
aWNlcw0KPiANCj4gYXV0b21hZ2ljYWxseSBydW5uaW5nIGRtc2V0dXAgZGlyZWN0bHkgZnJvbSB1
ZGV2IHRvIGNyZWF0ZSBhIGRtLXpvbmVkDQo+IHRhcmdldCBpcyB2ZXJ5IG11Y2ggd3JvbmcuICBJ
dCBqdXN0IGdldHMgaW4gdGhlIHdheSBvZiBwcm9wZXIgc3VwcG9ydA0KPiB0aGF0IHNob3VsZCBi
ZSBhZGQgdG8gYXBwcm9wcmlhdGUgdG9vbHMgdGhhdCBhZG1pbnMgdXNlIHRvIHNldHVwIHRoZWly
DQo+IHpvbmVkIGRldmljZXMuICBGb3IgaW5zdGFuY2UsIHBlcnNpc3RlbnQgdXNlIG9mIGRtLXpv
bmVkIHRhcmdldCBzaG91bGQNCj4gYmUgbWFkZSByZWxpYWJsZSB3aXRoIGEgdm9sdW1lIG1hbmFn
ZXIuLg0KPiANCj4gSW4gZ2VuZXJhbCB0aGlzIHVkZXYgc2NyaXB0IGlzIHVud2VsY29tZSBhbmQg
bWFrZXMgdGhpbmdzIHdheSB3b3JzZSBmb3INCj4gdGhlIGxvbmctdGVybSBzdWNjZXNzIG9mIHpv
bmVkIGRldmljZXMuDQo+IA0KPiBJIGRvbid0IGRpc3B1dGUgdGhlcmUgaXMgYW4gb2J2aW91cyB2
b2lkIGZvciBob3cgdG8gcHJvcGVybHkgc2V0dXAgem9uZWQNCj4gZGV2aWNlcywgYnV0IHRoaXMg
c2NyaXB0IGlzIF9ub3RfIHdoYXQgc2hvdWxkIGZpbGwgdGhhdCB2b2lkLg0KDQpGYWlyIHBvaW50
cy4gSSBhZ3JlZSB0aGF0IGl0IGlzIGhhY2tpc2guIFRoZSBpbnRlbnQgd2FzIGFzIHlvdSBzYXkg
dG8NCnRlbXBvcmFyaWx5IGZpbGwgYSB2b2lkLiBCdXQgZ3JhbnRlZCwgdGVtcG9yYXJ5IGhhY2tz
IHRlbmQgdG8gc3RheSAodG9vKQ0KbG9uZyBhcm91bmQgYW5kIGNhbiBpbiB0aGUgZW5kIGdldCBp
biB0aGUgd2F5IG9mIGNsZWFuIHNvbHV0aW9ucy4gSSB3aWxsDQpzdGFydCBsb29raW5nIGludG8g
YSBwcm9wZXIgZml4IGZvciBkbS16b25lZCBzZXR1cCBwZXJzaXN0ZW5jZS4NCg0KPiBTbyBhIGhl
YXJ0ZmVsdDoNCj4gDQo+IE5hY2tlZC1ieTogTWlrZSBTbml0emVyIDxzbml0emVyQHJlZGhhdC5j
b20+DQoNClVuZGVyc3Rvb2QuIEkgd2lsbCByZXZlcnQgYW5kIG5vdCBkb2N1bWVudCB0aGlzIHVk
ZXYgcnVsZSBpbg0KZG0tem9uZWQtdG9vbHMuIEkgd2FzIGEgbGl0dGxlIHRvbyBxdWljayBpbiBh
cHBseWluZyB0aGlzIHBhdGNoIGFuZCBkaWQNCm5vdCB3YWl0IHRvIHNlZSBjb21tZW50cyBmaXJz
dC4gTXkgYmFkLg0KDQpUaGFuayB5b3UgZm9yIHlvdXIgY29tbWVudHMuDQoNCkJlc3QgcmVnYXJk
cy4NCg0KLS0gDQpEYW1pZW4gTGUgTW9hbCwNCldlc3Rlcm4gRGlnaXRhbA==

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

Mike,

On 6/14/18 21:38, Mike Snitzer wrote:
> On Wed, Jun 13 2018 at  8:11pm -0400,
> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> 
>> Setting up a zoned disks in a generic form is not so trivial. There
>> is also quite a bit of tribal knowledge with these devices which is not
>> easy to find.
>>
>> The currently supplied demo script works but it is not generic enough to be
>> practical for Linux distributions or even developers which often move
>> from one kernel to another.
>>
>> This tries to put a bit of this tribal knowledge into an initial udev
>> rule for development with the hopes Linux distributions can later
>> deploy. Three rule are added. One rule is optional for now, it should be
>> extended later to be more distribution-friendly and then I think this
>> may be ready for consideration for integration on distributions.
>>
>> 1) scheduler setup
> 
> This is wrong.. if zoned devices are so dependent on deadline or
> mq-deadline then the kernel should allow them to be hardcoded.  I know
> Jens removed the API to do so but the fact that drivers need to rely on
> hacks like this udev rule to get a functional device is proof we need to
> allow drivers to impose the scheduler used.

I agree. Switching scheduler in the kernel during device probe/bring-up
would be my preferred choice. But until we come to a consensus on the
best way to do this, I think that this udev rule is useful since the
"scheduler=" kernel parameter is rather heavy handed and applies to all
single queue devices. Not to mention that adding this parameter to the
kernel arguments is in essence similar to the udev rule addition: action
from the system user is necessary to achieve a correct configuration
even though that could easily be done automatically from within the
block layer.

In the meantime, documenting properly that the deadline scheduler has a
special relationship with zoned block devices is still a nice thing to
do. But yes, dm-zoned-tools may not be the best place to do that. A
simple text file under Documentation/block may be better. At the very
least, Documentation/block/deadline-iosched.txt should have some mention
of its almost mandatory use with zoned block devices. I will work on that.

>> 2) backlist f2fs devices
> 
> There should porbably be support in dm-zoned for detecting whether a
> zoned device was formatted with f2fs (assuming there is a known f2fs
> superblock)?

That would certainly be nice to have and would not be too hard to code
in dmzadm. I can add that and do what for instance mkfs.ext4 or mkfs.xfs
do, which is ask for confirmation or bail out if an existing valid FS
format is detected on the disk.

That said, such test are far from common. mdadm for instance will
happily format and start an array using unmounted disks with valid file
systems on them.

>> 3) run dmsetup for the rest of devices
> 
> automagically running dmsetup directly from udev to create a dm-zoned
> target is very much wrong.  It just gets in the way of proper support
> that should be add to appropriate tools that admins use to setup their
> zoned devices.  For instance, persistent use of dm-zoned target should
> be made reliable with a volume manager..
> 
> In general this udev script is unwelcome and makes things way worse for
> the long-term success of zoned devices.
> 
> I don't dispute there is an obvious void for how to properly setup zoned
> devices, but this script is _not_ what should fill that void.

Fair points. I agree that it is hackish. The intent was as you say to
temporarily fill a void. But granted, temporary hacks tend to stay (too)
long around and can in the end get in the way of clean solutions. I will
start looking into a proper fix for dm-zoned setup persistence.

> So a heartfelt:
> 
> Nacked-by: Mike Snitzer <snitzer@redhat.com>

Understood. I will revert and not document this udev rule in
dm-zoned-tools. I was a little too quick in applying this patch and did
not wait to see comments first. My bad.

Thank you for your comments.

Best regards.

-- 
Damien Le Moal,
Western Digital

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

TWlrZSwNCg0KT24gNi8xNS8xOCAwMjo1OCwgTWlrZSBTbml0emVyIHdyb3RlOg0KPiBPbiBUaHUs
IEp1biAxNCAyMDE4IGF0ICAxOjM3cG0gLTA0MDAsDQo+IEx1aXMgUi4gUm9kcmlndWV6IDxtY2dy
b2ZAa2VybmVsLm9yZz4gd3JvdGU6DQo+IA0KPj4gT24gVGh1LCBKdW4gMTQsIDIwMTggYXQgMDg6
Mzg6MDZBTSAtMDQwMCwgTWlrZSBTbml0emVyIHdyb3RlOg0KPj4+IE9uIFdlZCwgSnVuIDEzIDIw
MTggYXQgIDg6MTFwbSAtMDQwMCwNCj4+PiBMdWlzIFIuIFJvZHJpZ3VleiA8bWNncm9mQGtlcm5l
bC5vcmc+IHdyb3RlOg0KPj4+DQo+Pj4+IFNldHRpbmcgdXAgYSB6b25lZCBkaXNrcyBpbiBhIGdl
bmVyaWMgZm9ybSBpcyBub3Qgc28gdHJpdmlhbC4gVGhlcmUNCj4+Pj4gaXMgYWxzbyBxdWl0ZSBh
IGJpdCBvZiB0cmliYWwga25vd2xlZGdlIHdpdGggdGhlc2UgZGV2aWNlcyB3aGljaCBpcyBub3QN
Cj4+Pj4gZWFzeSB0byBmaW5kLg0KPj4+Pg0KPj4+PiBUaGUgY3VycmVudGx5IHN1cHBsaWVkIGRl
bW8gc2NyaXB0IHdvcmtzIGJ1dCBpdCBpcyBub3QgZ2VuZXJpYyBlbm91Z2ggdG8gYmUNCj4+Pj4g
cHJhY3RpY2FsIGZvciBMaW51eCBkaXN0cmlidXRpb25zIG9yIGV2ZW4gZGV2ZWxvcGVycyB3aGlj
aCBvZnRlbiBtb3ZlDQo+Pj4+IGZyb20gb25lIGtlcm5lbCB0byBhbm90aGVyLg0KPj4+Pg0KPj4+
PiBUaGlzIHRyaWVzIHRvIHB1dCBhIGJpdCBvZiB0aGlzIHRyaWJhbCBrbm93bGVkZ2UgaW50byBh
biBpbml0aWFsIHVkZXYNCj4+Pj4gcnVsZSBmb3IgZGV2ZWxvcG1lbnQgd2l0aCB0aGUgaG9wZXMg
TGludXggZGlzdHJpYnV0aW9ucyBjYW4gbGF0ZXINCj4+Pj4gZGVwbG95LiBUaHJlZSBydWxlIGFy
ZSBhZGRlZC4gT25lIHJ1bGUgaXMgb3B0aW9uYWwgZm9yIG5vdywgaXQgc2hvdWxkIGJlDQo+Pj4+
IGV4dGVuZGVkIGxhdGVyIHRvIGJlIG1vcmUgZGlzdHJpYnV0aW9uLWZyaWVuZGx5IGFuZCB0aGVu
IEkgdGhpbmsgdGhpcw0KPj4+PiBtYXkgYmUgcmVhZHkgZm9yIGNvbnNpZGVyYXRpb24gZm9yIGlu
dGVncmF0aW9uIG9uIGRpc3RyaWJ1dGlvbnMuDQo+Pj4+DQo+Pj4+IDEpIHNjaGVkdWxlciBzZXR1
cA0KPj4+DQo+Pj4gVGhpcyBpcyB3cm9uZy4uIGlmIHpvbmVkIGRldmljZXMgYXJlIHNvIGRlcGVu
ZGVudCBvbiBkZWFkbGluZSBvcg0KPj4+IG1xLWRlYWRsaW5lIHRoZW4gdGhlIGtlcm5lbCBzaG91
bGQgYWxsb3cgdGhlbSB0byBiZSBoYXJkY29kZWQuICBJIGtub3cNCj4+PiBKZW5zIHJlbW92ZWQg
dGhlIEFQSSB0byBkbyBzbyBidXQgdGhlIGZhY3QgdGhhdCBkcml2ZXJzIG5lZWQgdG8gcmVseSBv
bg0KPj4+IGhhY2tzIGxpa2UgdGhpcyB1ZGV2IHJ1bGUgdG8gZ2V0IGEgZnVuY3Rpb25hbCBkZXZp
Y2UgaXMgcHJvb2Ygd2UgbmVlZCB0bw0KPj4+IGFsbG93IGRyaXZlcnMgdG8gaW1wb3NlIHRoZSBz
Y2hlZHVsZXIgdXNlZC4NCj4+DQo+PiBUaGlzIGlzIHRoZSBwb2ludCB0byB0aGUgcGF0Y2ggYXMg
d2VsbCwgSSBhY3R1YWxseSB0ZW5kIHRvIGFncmVlIHdpdGggeW91LA0KPj4gYW5kIEkgaGFkIHRy
aWVkIHRvIGRyYXcgdXAgYSBwYXRjaCB0byBkbyBqdXN0IHRoYXQsIGhvd2V2ZXIgaXRzICpub3Qq
IHBvc3NpYmxlDQo+PiB0b2RheSB0byBkbyB0aGlzIGFuZCB3b3VsZCByZXF1aXJlIHNvbWUgY29u
c2Vuc3VzLiBTbyBmcm9tIHdoYXQgSSBjYW4gdGVsbA0KPj4gd2UgKmhhdmUqIHRvIGxpdmUgd2l0
aCB0aGlzIG9uZSBvciBhIGZvcm0gb2YgaXQuIEllIGEgZmlsZSBkZXNjcmliaW5nIHdoaWNoDQo+
PiBkaXNrIHNlcmlhbCBnZXRzIGRlYWRsaW5lIGFuZCB3aGljaCBvbmUgZ2V0cyBtcS1kZWFkbGlu
ZS4NCj4+DQo+PiBKZW5zPw0KPj4NCj4+IEFueXdheSwgbGV0J3MgYXNzdW1lIHRoaXMgaXMgZG9u
ZSBpbiB0aGUga2VybmVsLCB3aGljaCBvbmUgd291bGQgdXNlIGRlYWRsaW5lLA0KPj4gd2hpY2gg
b25lIHdvdWxkIHVzZSBtcS1kZWFkbGluZT8NCj4gDQo+IFRoZSB6b25lZCBzdG9yYWdlIGRyaXZl
ciBuZWVkcyB0byBtYWtlIHRoYXQgY2FsbCBiYXNlZCBvbiB3aGF0IG1vZGUgaXQNCj4gaXMgaW4u
ICBJZiBpdCBpcyB1c2luZyBibGstbXEgdGhlbiBpdCBzZWxlY3RzIG1xLWRlYWRsaW5lLCBvdGhl
cndpc2UNCj4gZGVhZGxpbmUuDQoNCkFzIEJhcnQgcG9pbnRlZCBvdXQsIGRlYWRsaW5lIGlzIGFu
IGFsaWFzIG9mIG1xLWRlYWRsaW5lLiBTbyB1c2luZw0KImRlYWRsaW5lIiBhcyB0aGUgc2NoZWR1
bGVyIG5hbWUgd29ya3MgaW4gYm90aCBsZWdhY3kgYW5kIG1xIGNhc2VzLg0KDQo+Pj4+IDIpIGJh
Y2tsaXN0IGYyZnMgZGV2aWNlcw0KPj4+DQo+Pj4gVGhlcmUgc2hvdWxkIHBvcmJhYmx5IGJlIHN1
cHBvcnQgaW4gZG0tem9uZWQgZm9yIGRldGVjdGluZyB3aGV0aGVyIGENCj4+PiB6b25lZCBkZXZp
Y2Ugd2FzIGZvcm1hdHRlZCB3aXRoIGYyZnMgKGFzc3VtaW5nIHRoZXJlIGlzIGEga25vd24gZjJm
cw0KPj4+IHN1cGVyYmxvY2spPw0KPj4NCj4+IE5vdCBzdXJlIHdoYXQgeW91IG1lYW4uIEFyZSB5
b3Ugc3VnZ2VzdGluZyB3ZSBhbHdheXMgc2V0dXAgZG0tem9uZWQgZm9yDQo+PiBhbGwgem9uZWQg
ZGlza3MgYW5kIGp1c3QgbWFrZSBhbiBleGNlbXB0aW9uIG9uIGRtLXpvbmUgY29kZSB0byBzb21l
aG93DQo+PiB1c2UgdGhlIGRpc2sgZGlyZWN0bHkgaWYgYSBmaWxlc3lzdGVtIHN1cHBvcnRzIHpv
bmVkIGRpc2tzIGRpcmVjdGx5IHNvbWVob3c/DQo+IA0KPiBObywgSSdtIHNheWluZyB0aGF0IGEg
dWRldiBydWxlIHdvdWxkbid0IGJlIG5lZWRlZCBpZiBkbS16b25lZCBqdXN0DQo+IGVycm9yZWQg
b3V0IGlmIGFza2VkIHRvIGNvbnN1bWUgZGlza3MgdGhhdCBhbHJlYWR5IGhhdmUgYW4gZjJmcw0K
PiBzdXBlcmJsb2NrLiAgQW5kIGV4aXN0aW5nIGZpbGVzeXN0ZW1zIHNob3VsZCBnZXQgY29uZmxp
Y3Rpbmcgc3VwZXJibG9jaw0KPiBhd2FyZW5lc3MgImZvciBmcmVlIiBpZiBibGtpZCBvciB3aGF0
ZXZlciBpcyB0cmFpbmVkIHRvIGJlIGF3YXJlIG9mDQo+IGYyZnMncyBzdXBlcmJsb2NrLg0KDQpX
ZWxsIHRoYXQgaXMgdGhlIGNhc2UgYWxyZWFkeTogb24gc3RhcnR1cCwgZG0tem9uZWQgd2lsbCBy
ZWFkIGl0cyBvd24NCm1ldGFkYXRhIGZyb20gc2VjdG9yIDAsIHNhbWUgYXMgZjJmcyB3b3VsZCBk
byB3aXRoIGl0cyBzdXBlci1ibG9jay4gSWYNCnRoZSBmb3JtYXQvbWFnaWMgZG9lcyBub3QgbWF0
Y2ggZXhwZWN0ZWQgdmFsdWVzLCBkbS16b25lZCB3aWxsIGJhaWwgb3V0DQphbmQgcmV0dXJuIGFu
IGVycm9yLiBkbS16b25lZCBtZXRhZGF0YSBhbmQgZjJmcyBtZXRhZGF0YSByZXNpZGUgaW4gdGhl
DQpzYW1lIHBsYWNlIGFuZCBvdmVyd3JpdGUgZWFjaCBvdGhlci4gVGhlcmUgaXMgbm8gd2F5IHRv
IGdldCBvbmUgd29ya2luZw0Kb24gdG9wIG9mIHRoZSBvdGhlci4gSSBkbyBub3Qgc2VlIGFueSBw
b3NzaWJpbGl0eSBvZiBhIHByb2JsZW0gb24gc3RhcnR1cC4NCg0KQnV0IGRlZmluaXRlbHksIHRo
ZSB1c2VyIGxhbmQgZm9ybWF0IHRvb2xzIGNhbiBzdGVwIG9uIGVhY2ggb3RoZXIgdG9lcy4NClRo
YXQgbmVlZHMgZml4aW5nLg0KDQo+PiBmMmZzIGRvZXMgbm90IHJlcXVpcmUgZG0tem9uZWQuIFdo
YXQgd291bGQgYmUgcmVxdWlyZWQgaXMgYSBiaXQgbW9yZSBjb21wbGV4DQo+PiBnaXZlbiBvbmUg
Y291bGQgZGVkaWNhdGUgcG9ydGlvbnMgb2YgdGhlIGRpc2sgdG8gZjJmcyBhbmQgb3RoZXIgcG9y
dGlvbnMgdG8NCj4+IGFub3RoZXIgZmlsZXN5c3RlbSwgd2hpY2ggd291bGQgcmVxdWlyZSBkbS16
b25lZC4NCj4+DQo+PiBBbHNvIGZpbGVzeXN0ZW1zIHdoaWNoICpkbyBub3QqIHN1cHBvcnQgem9u
ZWQgZGlza3Mgc2hvdWxkICpub3QqIGJlIGFsbG93aW5nDQo+PiBkaXJlY3Qgc2V0dXAuIFRvZGF5
IHRoYXQncyBhbGwgZmlsZXN5c3RlbXMgb3RoZXIgdGhhbiBmMmZzLCBpbiB0aGUgZnV0dXJlDQo+
PiB0aGF0IG1heSBjaGFuZ2UuIFRob3NlIGFyZSBidWxsZXRzIHdlIGFyZSBhbGxvd2luZyB0byB0
cmlnZ2VyIGZvciB1c2Vycw0KPj4ganVzdCB3YWl0aW5nIHRvIHNob3QgdGhlbXNlbHZlcyBvbiB0
aGUgZm9vdCB3aXRoLg0KPj4NCj4+IFNvIHdobydzIGdvaW5nIHRvIHdvcmsgb24gYWxsIHRoZSBh
Ym92ZT8NCj4gDQo+IEl0IHNob3VsZCB0YWtlIGNhcmUgb2YgaXRzZWxmIGlmIGV4aXN0aW5nIHRv
b2xzIGFyZSB0cmFpbmVkIHRvIGJlIGF3YXJlDQo+IG9mIG5ldyBzaWduYXR1cmVzLiAgRS5nLiBl
eHQ0IGFuZCB4ZnMgYWxyZWFkeSBhcmUgYXdhcmUgb2Ygb25lIGFub3RoZXINCj4gc28gdGhhdCB5
b3UgY2Fubm90IHJlZm9ybWF0IGEgZGV2aWNlIHdpdGggdGhlIG90aGVyIHVubGVzcyBmb3JjZSBp
cw0KPiBnaXZlbi4NCj4gDQo+IFNhbWUga2luZCBvZiBtdXR1YWwgZXhjbHVzc2lvbiBuZWVkcyB0
byBoYXBwZW4gZm9yIHpvbmVkIGRldmljZXMuDQoNClllcy4NCg0KPiBTbyB0aGUgem9uZWQgZGV2
aWNlIHRvb2xzLCBkbS16b25lZCwgZjJmcywgd2hhdGV2ZXIuLiB0aGV5IG5lZWQgdG8gYmUNCj4g
dXBkYXRlZCB0byBub3Qgc3RlcCBvbiBlYWNoIG90aGVycyB0b2VzLiAgQW5kIG90aGVyIGZpbGVz
eXN0ZW1zJyB0b29scw0KPiBuZWVkIHRvIGJlIHVwZGF0ZWQgdG8gYmUgem9uZWQgZGV2aWNlIGF3
YXJlLg0KDQpJIHdpbGwgdXBkYXRlIGRtLXpvbmVkIHRvb2xzIHRvIGNoZWNrIGZvciBrbm93biBG
UyBzdXBlcmJsb2NrcywNCnNpbWlsYXJseSB0byB3aGF0IG1rZnMuZXh0NCBhbmQgbWtmcy54ZnMg
ZG8uDQoNCj4+Pj4gMykgcnVuIGRtc2V0dXAgZm9yIHRoZSByZXN0IG9mIGRldmljZXMNCj4+Pg0K
Pj4+IGF1dG9tYWdpY2FsbHkgcnVubmluZyBkbXNldHVwIGRpcmVjdGx5IGZyb20gdWRldiB0byBj
cmVhdGUgYSBkbS16b25lZA0KPj4+IHRhcmdldCBpcyB2ZXJ5IG11Y2ggd3JvbmcuICBJdCBqdXN0
IGdldHMgaW4gdGhlIHdheSBvZiBwcm9wZXIgc3VwcG9ydA0KPj4+IHRoYXQgc2hvdWxkIGJlIGFk
ZCB0byBhcHByb3ByaWF0ZSB0b29scyB0aGF0IGFkbWlucyB1c2UgdG8gc2V0dXAgdGhlaXINCj4+
PiB6b25lZCBkZXZpY2VzLiAgRm9yIGluc3RhbmNlLCBwZXJzaXN0ZW50IHVzZSBvZiBkbS16b25l
ZCB0YXJnZXQgc2hvdWxkDQo+Pj4gYmUgbWFkZSByZWxpYWJsZSB3aXRoIGEgdm9sdW1lIG1hbmFn
ZXIuLg0KPj4NCj4+IEFoIHllcywgYnV0IHdobydzIHdvcmtpbmcgb24gdGhhdD8gSG93IGxvbmcg
d2lsbCBpdCB0YWtlPw0KPiANCj4gTm8gaWRlYSwgYXMgaXMgKGZyb20gbXkgdmFudGFnZSBwb2lu
dCkgdGhlcmUgaXMgY2xvc2UgdG8gemVybyBkZW1hbmQgZm9yDQo+IHpvbmVkIGRldmljZXMuICBJ
dCB3b24ndCBiZSBhIHByaW9yaXR5IHVudGlsIGVub3VnaCBjdXN0b21lcnMgYXJlIGFza2luZw0K
PiBmb3IgaXQuDQoNCkZyb20gbXkgcG9pbnQgb2YgdmlldyAoZHJpdmUgdmVuZG9yKSwgdGhpbmdz
IGFyZSBkaWZmZXJlbnQuIFdlIGRvIHNlZSBhbg0KaW5jcmVhc2luZyBpbnRlcmVzdCBmb3IgdGhl
c2UgZHJpdmVzLiBIb3dldmVyLCBtb3N0IHVzZSBjYXNlcyBhcmUgc3RpbGwNCmxpbWl0ZWQgdG8g
YXBwbGljYXRpb24gYmFzZWQgZGlyZWN0IGRpc2sgYWNjZXNzIHdpdGggbWluaW1hbCBpbnZvbHZl
bWVudA0KZnJvbSB0aGUga2VybmVsIGFuZCBzbyBmZXcgInN1cHBvcnQiIHJlcXVlc3RzLiBNYW55
IHJlYXNvbnMgdG8gdGhpcywgYnV0DQpvbmUgaXMgdG8gc29tZSBleHRlbnQgdGhlIGN1cnJlbnQg
bGFjayBvZiBleHRlbmRlZCBzdXBwb3J0IGJ5IHRoZQ0Ka2VybmVsLiBEZXNwaXRlIGFsbCB0aGUg
cmVjZW50IHdvcmsgZG9uZSwgYXMgTHVpcyBleHBlcmllbmNlZCwgem9uZWQNCmRyaXZlcyBhcmUg
c3RpbGwgZmFyIGhhcmRlciB0byBlYXNpbHkgc2V0dXAgdGhhbiByZWd1bGFyIGRpc2tzLiBDaGlj
a2VuDQphbmQgZWdnIHNpdHVhdGlvbi4uLg0KDQo+PiBJIGFncmVlIGl0IGlzIG9kZCB0byBleHBl
Y3Qgb25lIHRvIHVzZSBkbXNldHVwIGFuZCB0aGVuIHVzZSBhIHZvbHVtZSBtYW5hZ2VyIG9uDQo+
PiB0b3Agb2YgaXQsIGlmIHdlIGNhbiBqdXN0IGFkZCBwcm9wZXIgc3VwcG9ydCBvbnRvIHRoZSB2
b2x1bWUgbWFuYWdlci4uLiB0aGVuDQo+PiB0aGF0J3MgYSByZWFzb25hYmxlIHdheSB0byBnby4N
Cj4+DQo+PiBCdXQgKndlJ3JlIG5vdCB0aGVyZSogeWV0LCBhbmQgYXMtaXMgdG9kYXksIHdoYXQg
aXMgZGVzY3JpYmVkIGluIHRoZSB1ZGV2DQo+PiBzY3JpcHQgaXMgdGhlIGJlc3Qgd2UgY2FuIGRv
IGZvciBhIGdlbmVyaWMgc2V0dXAuDQo+IA0KPiBKdXN0IGJlY2F1c2UgZG9pbmcgdGhpbmdzIHJp
Z2h0IHRha2VzIHdvcmsgZG9lc24ndCBtZWFuIGl0IG1ha2VzIHNlbnNlDQo+IHRvIGVsZXZhdGUg
dGhpcyB1ZGV2IHNjcmlwdCB0byBiZSBwYWNrYWdlZCBpbiBzb21lIHVwc3RyZWFtIHByb2plY3Qg
bGlrZQ0KPiB1ZGV2IG9yIHdoYXRldmVyLg0KDQpBZ3JlZS4gV2lsbCBzdGFydCBsb29raW5nIGlu
dG8gYmV0dGVyIHNvbHV0aW9ucyBub3cgdGhhdCBhdCBsZWFzdCBvbmUNCnVzZXIgKEx1aXMpIGNv
bXBsYWluZWQuIFRoZSBjdXN0b21lciBpcyBraW5nLg0KDQo+Pj4gSW4gZ2VuZXJhbCB0aGlzIHVk
ZXYgc2NyaXB0IGlzIHVud2VsY29tZSBhbmQgbWFrZXMgdGhpbmdzIHdheSB3b3JzZSBmb3INCj4+
PiB0aGUgbG9uZy10ZXJtIHN1Y2Nlc3Mgb2Ygem9uZWQgZGV2aWNlcy4NCj4+DQo+PiBkbS16b25l
ZC10b29scyBkb2VzIG5vdCBhY2tub3dsZWRnZSBpbiBhbnkgd2F5IGEgcm9hZG1hcCwgYW5kIGp1
c3QgcHJvdmlkZXMNCj4+IGEgc2NyaXB0LCB3aGljaCBJTUhPIGlzIGxlc3MgZ2VuZXJpYyBhbmQg
bGVzcyBkaXN0cmlidXRpb24gZnJpZW5kbHkuIEhhdmluZw0KPj4gYSB1ZGV2IHJ1bGUgaW4gcGxh
Y2UgdG8gZGVtb25zdHJhdGUgdGhlIGN1cnJlbnQgc3RhdGUgb2YgYWZmYWlycyBJTUhPIGlzDQo+
PiBtb3JlIHNjYWxhYmxlIGRlbW9uc3RyYXRlcyB0aGUgaXNzdWVzIGJldHRlciB0aGFuIHRoZSBz
Y3JpcHQuDQo+Pg0KPj4gSWYgd2UgaGF2ZSBhbiBhZ3JlZWQgdXBvbiBsb25nIHRlcm0gc3RyYXRl
Z3kgbGV0cyBkb2N1bWVudCB0aGF0LiBCdXQgZnJvbQ0KPj4gd2hhdCBJIGdhdGhlciB3ZSBhcmUg
bm90IGV2ZW4gaW4gY29uc2Vuc3VzIHdpdGggcmVnYXJkcyB0byB0aGUgc2NoZWR1bGVyDQo+PiBz
dHVmZi4gSWYgd2UgaGF2ZSBjb25zZW5zdXMgb24gdGhlIG90aGVyIHN0dWZmIGxldHMgZG9jdW1l
bnQgdGhhdCBhcw0KPj4gZG0tem9uZWQtdG9vbHMgaXMgdGhlIG9ubHkgcGxhY2UgSSB0aGluayBm
b2xrcyBjb3VsZCBmaW5kIHRvIHJlYXNvbmFibHkNCj4+IGRlcGxveSB0aGVzZSB0aGluZ3MuDQo+
IA0KPiBJJ20gc3VyZSBEYW1pZW4gYW5kIG90aGVycyB3aWxsIGhhdmUgc29tZXRoaW5nIHRvIHNh
eSBoZXJlLg0KDQpZZXMuIFRoZSBzY2hlZHVsZXIgc2V0dXAgcGFpbiBpcyByZWFsLiBKZW5zIG1h
ZGUgaXQgY2xlYXIgdGhhdCBoZQ0KcHJlZmVycyBhIHVkZXYgcnVsZS4gSSBmdWxseSB1bmRlcnN0
YW5kIGhpcyBwb2ludCBvZiB2aWV3LCB5ZXQsIEkgdGhpbmsNCmFuIGF1dG9tYXRpYyBzd2l0Y2gg
aW4gdGhlIGJsb2NrIGxheWVyIHdvdWxkIGJlIGZhciBlYXNpZXIgYW5kIGdlbmVyYXRlDQphIGxv
dCBsZXNzIHByb2JsZW0gZm9yIHVzZXJzLCBhbmQgbGlrZWx5IGxlc3MgImJ1ZyByZXBvcnQiIHRv
DQpkaXN0cmlidXRpb25zIHZlbmRvcnMgKGFuZCB0byBteXNlbGYgdG9vKS4NCg0KVGhhdCBzYWlk
LCBJIGFsc28gbGlrZSB0byBzZWUgdGhlIGN1cnJlbnQgZGVwZW5kZW5jeSBvZiB6b25lZCBkZXZp
Y2VzIG9uDQp0aGUgZGVhZGxpbmUgc2NoZWR1bGVyIGFzIHRlbXBvcmFyeSB1bnRpbCBhIGJldHRl
ciBzb2x1dGlvbiBmb3IgZW5zdXJpbmcNCndyaXRlIG9yZGVyaW5nIGlzIGZvdW5kLiBBZnRlciBh
bGwsIHJlcXVpcmluZyBkZWFkbGluZSBhcyB0aGUgZGlzaw0Kc2NoZWR1bGVyIGRvZXMgaW1wb3Nl
IG90aGVyIGxpbWl0YXRpb25zIG9uIHRoZSB1c2VyLiBMYWNrIG9mIEkvTw0KcHJpb3JpdHkgc3Vw
cG9ydCBhbmQgbm8gY2dyb3VwIGJhc2VkIGZhaXJuZXNzIGFyZSB0d28gZXhhbXBsZXMgb2Ygd2hh
dA0Kb3RoZXIgc2NoZWR1bGVycyBwcm92aWRlIGJ1dCBpcyBsb3N0IHdpdGggZm9yY2luZyBkZWFk
bGluZS4NCg0KVGhlIG9idmlvdXMgZml4IGlzIG9mIGNvdXJzZSB0byBtYWtlIGFsbCBkaXNrIHNj
aGVkdWxlcnMgem9uZSBkZXZpY2UNCmF3YXJlLiBBIGxpdHRsZSBoZWF2eSBoYW5kZWQsIHByb2Jh
Ymx5IGxvdHMgb2YgZHVwbGljYXRlZC9zaW1pbGFyIGNvZGUsDQphbmQgbWFueSBtb3JlIHRlc3Qg
Y2FzZXMgdG8gY292ZXIuIFRoaXMgYXBwcm9hY2ggZG9lcyBub3Qgc2VlbQ0Kc3VzdGFpbmFibGUg
dG8gbWUuDQoNCldlIGRpc2N1c3NlZCBvdGhlciBwb3NzaWJpbGl0aWVzIGF0IExTRi9NTSAoc3Bl
Y2lhbGl6ZWQgd3JpdGUgcXVldWUgaW4NCm11bHRpLXF1ZXVlIHBhdGgpLiBPbmUgY291bGQgYWxz
byB0aGluayBvZiBtb3JlIGludmFzaXZlIGNoYW5nZXMgdG8gdGhlDQpibG9jayBsYXllciAoZS5n
LiBhZGRpbmcgYW4gb3B0aW9uYWwgImRpc3BhdGNoZXIiIGxheWVyIHRvIHRpZ2h0bHkNCmNvbnRy
b2wgY29tbWFuZCBvcmRlcmluZyA/KS4gQW5kIHByb2JhYmx5IGEgbG90IG1vcmUgb3B0aW9ucywg
QnV0IEkgYW0NCm5vdCB5ZXQgc3VyZSB3aGF0IGFuIGFwcHJvcHJpYXRlIHJlcGxhY2VtZW50IHRv
IGRlYWRsaW5lIHdvdWxkIGJlLg0KDQpFdmVudHVhbGx5LCB0aGUgcmVtb3ZhbCBvZiB0aGUgbGVn
YWN5IEkvTyBwYXRoIG1heSBhbHNvIGJlIHRoZSB0cmlnZ2VyDQp0byBpbnRyb2R1Y2Ugc29tZSBk
ZWVwZXIgZGVzaWduIGNoYW5nZXMgdG8gYmxrLW1xIHRvIGFjY29tbW9kYXRlIG1vcmUNCmVhc2ls
eSB6b25lZCBibG9jayBkZXZpY2VzIG9yIG90aGVyIG5vbi1zdGFuZGFyZCBibG9jayBkZXZpY2Vz
IChvcGVuDQpjaGFubmVsIFNTRHMgZm9yIGluc3RhbmNlKS4NCg0KQXMgeW91IGNhbiBzZWUgZnJv
bSB0aGUgYWJvdmUsIHdvcmtpbmcgd2l0aCB0aGVzZSBkcml2ZXMgYWxsIGRheSBsb25nDQpkb2Vz
IG5vdCBtYWtlIGZvciBhIGNsZWFyIHN0cmF0ZWd5LiBJbnB1dHMgZnJvbSBvdGhlciBoZXJlIGFy
ZSBtb3JlIHRoYW4NCndlbGNvbWUuIEkgd291bGQgYmUgaGFwcHkgdG8gd3JpdGUgdXAgYWxsIHRo
ZSBpZGVhcyBJIGhhdmUgdG8gc3RhcnQgYQ0KZGlzY3Vzc2lvbiBzbyB0aGF0IHdlIGNhbiBjb21l
IHRvIGEgY29uc2Vuc3VzIGFuZCBoYXZlIGEgcGxhbi4NCg0KPj4+IEkgZG9uJ3QgZGlzcHV0ZSB0
aGVyZSBpcyBhbiBvYnZpb3VzIHZvaWQgZm9yIGhvdyB0byBwcm9wZXJseSBzZXR1cCB6b25lZA0K
Pj4+IGRldmljZXMsIGJ1dCB0aGlzIHNjcmlwdCBpcyBfbm90XyB3aGF0IHNob3VsZCBmaWxsIHRo
YXQgdm9pZC4NCj4+DQo+PiBHb29kIHRvIGtub3chIEFnYWluLCBjb25zaWRlciBpdCBhcyBhbiBh
bHRlcm5hdGl2ZSB0byB0aGUgc2NyaXB0Lg0KPj4NCj4+IEknbSBoYXBweSB0byBhZGFwdCB0aGUg
bGFuZ3VhZ2UgYW5kIHN1cHBseSBpdCBvbmx5IGFzIGFuIGV4YW1wbGUgc2NyaXB0DQo+PiBkZXZl
bG9wZXJzIGNhbiB1c2UsIGJ1dCB3ZSBjYW4ndCBsZWF2ZSB1c2VycyBoYW5naW5nIGFzIHdlbGwu
IExldCdzIGF0DQo+PiBsZWFzdCBjb21lIHVwIHdpdGggYSBwbGFuIHdoaWNoIHdlIHNlZW0gdG8g
YWdyZWUgb24gYW5kIGRvY3VtZW50IHRoYXQuDQo+IA0KPiBCZXN0IHRvIHRyeSB0byBnZXQgRGFt
aWVuIGFuZCBvdGhlcnMgbW9yZSBpbnZlc3RlZCBpbiB6b25lZCBkZXZpY2VzIHRvDQo+IGhlbHAg
eW91IHRha2UgdXAgeW91ciBjYXVzZS4gIEkgdGhpbmsgaXQgaXMgd29ydGh3aGlsZSB0byBkZXZl
bG9wIGENCj4gc3RyYXRlZ3kuICBCdXQgaXQgbmVlZHMgdG8gYmUgZG9uZSBpbiB0ZXJtcyBvZiB0
aGUgbm9ybXMgb2YgdGhlIGV4aXN0aW5nDQo+IGluZnJhc3RydWN0dXJlIHdlIGFsbCBtYWtlIHVz
ZSBvZiB0b2RheS4gIFNvIGZpcnN0IHN0ZXAgaXMgbWFraW5nDQo+IGV4aXN0aW5nIHRvb2xzIHpv
bmVkIGRldmljZSBhd2FyZSAoZXZlbiBpZiB0byByZWplY3Qgc3VjaCBkZXZpY2VzKS4NCg0KUmVz
dCBhc3N1cmVkIHRoYXQgSSBhbSBmdWxseSBpbnZlc3RlZCBpbiBpbXByb3ZpbmcgdGhlIGV4aXN0
aW5nDQppbmZyYXN0cnVjdHVyZSBmb3Igem9uZWQgYmxvY2sgZGV2aWNlcy4gQXMgbWVudGlvbmVk
IGFib3ZlLCBhcHBsaWNhdGlvbnMNCmJhc2VkIHVzZSBvZiB6b25lZCBibG9jayBkZXZpY2VzIHN0
aWxsIHByZXZhaWxzIHRvZGF5LiBTbyBJIGRvIHRlbmQgdG8NCndvcmsgbW9yZSBvbiB0aGF0IHNp
ZGUgb2YgdGhpbmdzIChsaWJ6YmMsIHRjbXUsIHN5c3V0aWxzIGZvciBpbnN0YW5jZSkNCnJhdGhl
ciB0aGFuIG9uIGEgYmV0dGVyIGludGVncmF0aW9uIHdpdGggbW9yZSBhZHZhbmNlZCB0b29scyAo
c3VjaCBhcw0KTFZNKSByZWx5aW5nIG9uIGtlcm5lbCBmZWF0dXJlcy4gSSBhbSBob3dldmVyIHNl
ZWluZyByaXNpbmcgaW50ZXJlc3QgaW4NCmZpbGUgc3lzdGVtcyBhbmQgYWxzbyBpbiBkbS16b25l
ZC4gU28gZGVmaW5pdGVseSBpdCBpcyB0aW1lIHRvIHN0ZXAgdXANCndvcmsgaW4gdGhhdCBhcmVh
IHRvIGZ1cnRoZXIgc2ltcGxpZnkgdXNpbmcgdGhlc2UgZHJpdmVzLg0KDQpUaGFuayB5b3UgZm9y
IHRoZSBmZWVkYmFjay4NCg0KQmVzdCByZWdhcmRzLg0KDQotLSANCkRhbWllbiBMZSBNb2FsLA0K
V2VzdGVybiBEaWdpdGFs

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Damien Le Moal]

Mike,

On 6/15/18 02:58, Mike Snitzer wrote:
> On Thu, Jun 14 2018 at  1:37pm -0400,
> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> 
>> On Thu, Jun 14, 2018 at 08:38:06AM -0400, Mike Snitzer wrote:
>>> On Wed, Jun 13 2018 at  8:11pm -0400,
>>> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
>>>
>>>> Setting up a zoned disks in a generic form is not so trivial. There
>>>> is also quite a bit of tribal knowledge with these devices which is not
>>>> easy to find.
>>>>
>>>> The currently supplied demo script works but it is not generic enough to be
>>>> practical for Linux distributions or even developers which often move
>>>> from one kernel to another.
>>>>
>>>> This tries to put a bit of this tribal knowledge into an initial udev
>>>> rule for development with the hopes Linux distributions can later
>>>> deploy. Three rule are added. One rule is optional for now, it should be
>>>> extended later to be more distribution-friendly and then I think this
>>>> may be ready for consideration for integration on distributions.
>>>>
>>>> 1) scheduler setup
>>>
>>> This is wrong.. if zoned devices are so dependent on deadline or
>>> mq-deadline then the kernel should allow them to be hardcoded.  I know
>>> Jens removed the API to do so but the fact that drivers need to rely on
>>> hacks like this udev rule to get a functional device is proof we need to
>>> allow drivers to impose the scheduler used.
>>
>> This is the point to the patch as well, I actually tend to agree with you,
>> and I had tried to draw up a patch to do just that, however its *not* possible
>> today to do this and would require some consensus. So from what I can tell
>> we *have* to live with this one or a form of it. Ie a file describing which
>> disk serial gets deadline and which one gets mq-deadline.
>>
>> Jens?
>>
>> Anyway, let's assume this is done in the kernel, which one would use deadline,
>> which one would use mq-deadline?
> 
> The zoned storage driver needs to make that call based on what mode it
> is in.  If it is using blk-mq then it selects mq-deadline, otherwise
> deadline.

As Bart pointed out, deadline is an alias of mq-deadline. So using
"deadline" as the scheduler name works in both legacy and mq cases.

>>>> 2) backlist f2fs devices
>>>
>>> There should porbably be support in dm-zoned for detecting whether a
>>> zoned device was formatted with f2fs (assuming there is a known f2fs
>>> superblock)?
>>
>> Not sure what you mean. Are you suggesting we always setup dm-zoned for
>> all zoned disks and just make an excemption on dm-zone code to somehow
>> use the disk directly if a filesystem supports zoned disks directly somehow?
> 
> No, I'm saying that a udev rule wouldn't be needed if dm-zoned just
> errored out if asked to consume disks that already have an f2fs
> superblock.  And existing filesystems should get conflicting superblock
> awareness "for free" if blkid or whatever is trained to be aware of
> f2fs's superblock.

Well that is the case already: on startup, dm-zoned will read its own
metadata from sector 0, same as f2fs would do with its super-block. If
the format/magic does not match expected values, dm-zoned will bail out
and return an error. dm-zoned metadata and f2fs metadata reside in the
same place and overwrite each other. There is no way to get one working
on top of the other. I do not see any possibility of a problem on startup.

But definitely, the user land format tools can step on each other toes.
That needs fixing.

>> f2fs does not require dm-zoned. What would be required is a bit more complex
>> given one could dedicate portions of the disk to f2fs and other portions to
>> another filesystem, which would require dm-zoned.
>>
>> Also filesystems which *do not* support zoned disks should *not* be allowing
>> direct setup. Today that's all filesystems other than f2fs, in the future
>> that may change. Those are bullets we are allowing to trigger for users
>> just waiting to shot themselves on the foot with.
>>
>> So who's going to work on all the above?
> 
> It should take care of itself if existing tools are trained to be aware
> of new signatures.  E.g. ext4 and xfs already are aware of one another
> so that you cannot reformat a device with the other unless force is
> given.
> 
> Same kind of mutual exclussion needs to happen for zoned devices.

Yes.

> So the zoned device tools, dm-zoned, f2fs, whatever.. they need to be
> updated to not step on each others toes.  And other filesystems' tools
> need to be updated to be zoned device aware.

I will update dm-zoned tools to check for known FS superblocks,
similarly to what mkfs.ext4 and mkfs.xfs do.

>>>> 3) run dmsetup for the rest of devices
>>>
>>> automagically running dmsetup directly from udev to create a dm-zoned
>>> target is very much wrong.  It just gets in the way of proper support
>>> that should be add to appropriate tools that admins use to setup their
>>> zoned devices.  For instance, persistent use of dm-zoned target should
>>> be made reliable with a volume manager..
>>
>> Ah yes, but who's working on that? How long will it take?
> 
> No idea, as is (from my vantage point) there is close to zero demand for
> zoned devices.  It won't be a priority until enough customers are asking
> for it.

From my point of view (drive vendor), things are different. We do see an
increasing interest for these drives. However, most use cases are still
limited to application based direct disk access with minimal involvement
from the kernel and so few "support" requests. Many reasons to this, but
one is to some extent the current lack of extended support by the
kernel. Despite all the recent work done, as Luis experienced, zoned
drives are still far harder to easily setup than regular disks. Chicken
and egg situation...

>> I agree it is odd to expect one to use dmsetup and then use a volume manager on
>> top of it, if we can just add proper support onto the volume manager... then
>> that's a reasonable way to go.
>>
>> But *we're not there* yet, and as-is today, what is described in the udev
>> script is the best we can do for a generic setup.
> 
> Just because doing things right takes work doesn't mean it makes sense
> to elevate this udev script to be packaged in some upstream project like
> udev or whatever.

Agree. Will start looking into better solutions now that at least one
user (Luis) complained. The customer is king.

>>> In general this udev script is unwelcome and makes things way worse for
>>> the long-term success of zoned devices.
>>
>> dm-zoned-tools does not acknowledge in any way a roadmap, and just provides
>> a script, which IMHO is less generic and less distribution friendly. Having
>> a udev rule in place to demonstrate the current state of affairs IMHO is
>> more scalable demonstrates the issues better than the script.
>>
>> If we have an agreed upon long term strategy lets document that. But from
>> what I gather we are not even in consensus with regards to the scheduler
>> stuff. If we have consensus on the other stuff lets document that as
>> dm-zoned-tools is the only place I think folks could find to reasonably
>> deploy these things.
> 
> I'm sure Damien and others will have something to say here.

Yes. The scheduler setup pain is real. Jens made it clear that he
prefers a udev rule. I fully understand his point of view, yet, I think
an automatic switch in the block layer would be far easier and generate
a lot less problem for users, and likely less "bug report" to
distributions vendors (and to myself too).

That said, I also like to see the current dependency of zoned devices on
the deadline scheduler as temporary until a better solution for ensuring
write ordering is found. After all, requiring deadline as the disk
scheduler does impose other limitations on the user. Lack of I/O
priority support and no cgroup based fairness are two examples of what
other schedulers provide but is lost with forcing deadline.

The obvious fix is of course to make all disk schedulers zone device
aware. A little heavy handed, probably lots of duplicated/similar code,
and many more test cases to cover. This approach does not seem
sustainable to me.

We discussed other possibilities at LSF/MM (specialized write queue in
multi-queue path). One could also think of more invasive changes to the
block layer (e.g. adding an optional "dispatcher" layer to tightly
control command ordering ?). And probably a lot more options, But I am
not yet sure what an appropriate replacement to deadline would be.

Eventually, the removal of the legacy I/O path may also be the trigger
to introduce some deeper design changes to blk-mq to accommodate more
easily zoned block devices or other non-standard block devices (open
channel SSDs for instance).

As you can see from the above, working with these drives all day long
does not make for a clear strategy. Inputs from other here are more than
welcome. I would be happy to write up all the ideas I have to start a
discussion so that we can come to a consensus and have a plan.

>>> I don't dispute there is an obvious void for how to properly setup zoned
>>> devices, but this script is _not_ what should fill that void.
>>
>> Good to know! Again, consider it as an alternative to the script.
>>
>> I'm happy to adapt the language and supply it only as an example script
>> developers can use, but we can't leave users hanging as well. Let's at
>> least come up with a plan which we seem to agree on and document that.
> 
> Best to try to get Damien and others more invested in zoned devices to
> help you take up your cause.  I think it is worthwhile to develop a
> strategy.  But it needs to be done in terms of the norms of the existing
> infrastructure we all make use of today.  So first step is making
> existing tools zoned device aware (even if to reject such devices).

Rest assured that I am fully invested in improving the existing
infrastructure for zoned block devices. As mentioned above, applications
based use of zoned block devices still prevails today. So I do tend to
work more on that side of things (libzbc, tcmu, sysutils for instance)
rather than on a better integration with more advanced tools (such as
LVM) relying on kernel features. I am however seeing rising interest in
file systems and also in dm-zoned. So definitely it is time to step up
work in that area to further simplify using these drives.

Thank you for the feedback.

Best regards.

-- 
Damien Le Moal,
Western Digital

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Martin Wilck]

On Thu, 2018-06-14 at 06:42 -0700, Christoph Hellwig wrote:
> On Thu, Jun 14, 2018 at 01:39:50PM +0000, Bart Van Assche wrote:
> > On Thu, 2018-06-14 at 10:01 +0000, Damien Le Moal wrote:
> > > Applied. Thanks Luis !
> > 
> > Hello Damien,
> > 
> > Can this still be undone? I agree with Mike that it's wrong to
> > invoke
> > "/sbin/dmsetup create ... zoned ..." from a udev rule.
> 
> Yes.  We'll really need to verfify the device has dm-zoned metadata
> first.  Preferably including a uuid for stable device naming.

libblkid would be the central hub for metadata discovery, so perhaps a
patch should be made to make libblkid dm-zoned-aware.

Anyway, as Damien explained, dmzoned bails out if it doesn't find
matching meta data, so AFAICS, little harm is done by calling it for a
SMR device in host-managed mode. I fail to get the point why this would
be wrong in general - what's the difference to e.g. calling "mdadm -I"?

Regards
Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

Re: [PATCH] dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Martin Wilck]

On Thu, 2018-06-14 at 06:42 -0700, Christoph Hellwig wrote:
> On Thu, Jun 14, 2018 at 01:39:50PM +0000, Bart Van Assche wrote:
> > On Thu, 2018-06-14 at 10:01 +0000, Damien Le Moal wrote:
> > > Applied. Thanks Luis !
> > 
> > Hello Damien,
> > 
> > Can this still be undone? I agree with Mike that it's wrong to
> > invoke
> > "/sbin/dmsetup create ... zoned ..." from a udev rule.
> 
> Yes.  We'll really need to verfify the device has dm-zoned metadata
> first.  Preferably including a uuid for stable device naming.

libblkid would be the central hub for metadata discovery, so perhaps a
patch should be made to make libblkid dm-zoned-aware.

Anyway, as Damien explained, dmzoned bails out if it doesn't find
matching meta data, so AFAICS, little harm is done by calling it for a
SMR device in host-managed mode. I fail to get the point why this would
be wrong in general - what's the difference to e.g. calling "mdadm -I"?

Regards
Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

Re: dm-zoned-tools: add zoned disk udev rules for scheduler / dmsetup

[Mike Snitzer]

On Fri, Jun 15 2018 at  5:59am -0400,
Damien Le Moal <Damien.LeMoal@wdc.com> wrote:

> Mike,
> 
> On 6/15/18 02:58, Mike Snitzer wrote:
> > On Thu, Jun 14 2018 at  1:37pm -0400,
> > Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> > 
> >> On Thu, Jun 14, 2018 at 08:38:06AM -0400, Mike Snitzer wrote:
> >>> On Wed, Jun 13 2018 at  8:11pm -0400,
> >>> Luis R. Rodriguez <mcgrof@kernel.org> wrote:
> >>>
> >>>> Setting up a zoned disks in a generic form is not so trivial. There
> >>>> is also quite a bit of tribal knowledge with these devices which is not
> >>>> easy to find.
> >>>>
> >>>> The currently supplied demo script works but it is not generic enough to be
> >>>> practical for Linux distributions or even developers which often move
> >>>> from one kernel to another.
> >>>>
> >>>> This tries to put a bit of this tribal knowledge into an initial udev
> >>>> rule for development with the hopes Linux distributions can later
> >>>> deploy. Three rule are added. One rule is optional for now, it should be
> >>>> extended later to be more distribution-friendly and then I think this
> >>>> may be ready for consideration for integration on distributions.
> >>>>
> >>>> 1) scheduler setup
> >>>
> >>> This is wrong.. if zoned devices are so dependent on deadline or
> >>> mq-deadline then the kernel should allow them to be hardcoded.  I know
> >>> Jens removed the API to do so but the fact that drivers need to rely on
> >>> hacks like this udev rule to get a functional device is proof we need to
> >>> allow drivers to impose the scheduler used.
> >>
> >> This is the point to the patch as well, I actually tend to agree with you,
> >> and I had tried to draw up a patch to do just that, however its *not* possible
> >> today to do this and would require some consensus. So from what I can tell
> >> we *have* to live with this one or a form of it. Ie a file describing which
> >> disk serial gets deadline and which one gets mq-deadline.
> >>
> >> Jens?
> >>
> >> Anyway, let's assume this is done in the kernel, which one would use deadline,
> >> which one would use mq-deadline?
> > 
> > The zoned storage driver needs to make that call based on what mode it
> > is in.  If it is using blk-mq then it selects mq-deadline, otherwise
> > deadline.
> 
> As Bart pointed out, deadline is an alias of mq-deadline. So using
> "deadline" as the scheduler name works in both legacy and mq cases.
> 
> >>>> 2) backlist f2fs devices
> >>>
> >>> There should porbably be support in dm-zoned for detecting whether a
> >>> zoned device was formatted with f2fs (assuming there is a known f2fs
> >>> superblock)?
> >>
> >> Not sure what you mean. Are you suggesting we always setup dm-zoned for
> >> all zoned disks and just make an excemption on dm-zone code to somehow
> >> use the disk directly if a filesystem supports zoned disks directly somehow?
> > 
> > No, I'm saying that a udev rule wouldn't be needed if dm-zoned just
> > errored out if asked to consume disks that already have an f2fs
> > superblock.  And existing filesystems should get conflicting superblock
> > awareness "for free" if blkid or whatever is trained to be aware of
> > f2fs's superblock.
> 
> Well that is the case already: on startup, dm-zoned will read its own
> metadata from sector 0, same as f2fs would do with its super-block. If
> the format/magic does not match expected values, dm-zoned will bail out
> and return an error. dm-zoned metadata and f2fs metadata reside in the
> same place and overwrite each other. There is no way to get one working
> on top of the other. I do not see any possibility of a problem on startup.
> 
> But definitely, the user land format tools can step on each other toes.
> That needs fixing.

Right, I was talking about in the .ctr path for initial device creation,
not activation of a previously created dm-zoned device.

But I agree it makes most sense to do this check in userspace.

> >> f2fs does not require dm-zoned. What would be required is a bit more complex
> >> given one could dedicate portions of the disk to f2fs and other portions to
> >> another filesystem, which would require dm-zoned.
> >>
> >> Also filesystems which *do not* support zoned disks should *not* be allowing
> >> direct setup. Today that's all filesystems other than f2fs, in the future
> >> that may change. Those are bullets we are allowing to trigger for users
> >> just waiting to shot themselves on the foot with.
> >>
> >> So who's going to work on all the above?
> > 
> > It should take care of itself if existing tools are trained to be aware
> > of new signatures.  E.g. ext4 and xfs already are aware of one another
> > so that you cannot reformat a device with the other unless force is
> > given.
> > 
> > Same kind of mutual exclussion needs to happen for zoned devices.
> 
> Yes.
> 
> > So the zoned device tools, dm-zoned, f2fs, whatever.. they need to be
> > updated to not step on each others toes.  And other filesystems' tools
> > need to be updated to be zoned device aware.
> 
> I will update dm-zoned tools to check for known FS superblocks,
> similarly to what mkfs.ext4 and mkfs.xfs do.

Thanks.

> >>>> 3) run dmsetup for the rest of devices
> >>>
> >>> automagically running dmsetup directly from udev to create a dm-zoned
> >>> target is very much wrong.  It just gets in the way of proper support
> >>> that should be add to appropriate tools that admins use to setup their
> >>> zoned devices.  For instance, persistent use of dm-zoned target should
> >>> be made reliable with a volume manager..
> >>
> >> Ah yes, but who's working on that? How long will it take?
> > 
> > No idea, as is (from my vantage point) there is close to zero demand for
> > zoned devices.  It won't be a priority until enough customers are asking
> > for it.
> 
> From my point of view (drive vendor), things are different. We do see an
> increasing interest for these drives. However, most use cases are still
> limited to application based direct disk access with minimal involvement
> from the kernel and so few "support" requests. Many reasons to this, but
> one is to some extent the current lack of extended support by the
> kernel. Despite all the recent work done, as Luis experienced, zoned
> drives are still far harder to easily setup than regular disks. Chicken
> and egg situation...
> 
> >> I agree it is odd to expect one to use dmsetup and then use a volume manager on
> >> top of it, if we can just add proper support onto the volume manager... then
> >> that's a reasonable way to go.
> >>
> >> But *we're not there* yet, and as-is today, what is described in the udev
> >> script is the best we can do for a generic setup.
> > 
> > Just because doing things right takes work doesn't mean it makes sense
> > to elevate this udev script to be packaged in some upstream project like
> > udev or whatever.
> 
> Agree. Will start looking into better solutions now that at least one
> user (Luis) complained. The customer is king.
> 
> >>> In general this udev script is unwelcome and makes things way worse for
> >>> the long-term success of zoned devices.
> >>
> >> dm-zoned-tools does not acknowledge in any way a roadmap, and just provides
> >> a script, which IMHO is less generic and less distribution friendly. Having
> >> a udev rule in place to demonstrate the current state of affairs IMHO is
> >> more scalable demonstrates the issues better than the script.
> >>
> >> If we have an agreed upon long term strategy lets document that. But from
> >> what I gather we are not even in consensus with regards to the scheduler
> >> stuff. If we have consensus on the other stuff lets document that as
> >> dm-zoned-tools is the only place I think folks could find to reasonably
> >> deploy these things.
> > 
> > I'm sure Damien and others will have something to say here.
> 
> Yes. The scheduler setup pain is real. Jens made it clear that he
> prefers a udev rule. I fully understand his point of view, yet, I think
> an automatic switch in the block layer would be far easier and generate
> a lot less problem for users, and likely less "bug report" to
> distributions vendors (and to myself too).

Yeap, Jens would say that ;)  Unfortnately using udev to get this
critical configuration correct is a real leap of faith that will prove
to be a whack-a-mole across distributions.

> That said, I also like to see the current dependency of zoned devices on
> the deadline scheduler as temporary until a better solution for ensuring
> write ordering is found. After all, requiring deadline as the disk
> scheduler does impose other limitations on the user. Lack of I/O
> priority support and no cgroup based fairness are two examples of what
> other schedulers provide but is lost with forcing deadline.
> 
> The obvious fix is of course to make all disk schedulers zone device
> aware. A little heavy handed, probably lots of duplicated/similar code,
> and many more test cases to cover. This approach does not seem
> sustainable to me.

Right, it isn't sustainable.  There isn't enough zoned device developer
expertise to go around.

> We discussed other possibilities at LSF/MM (specialized write queue in
> multi-queue path). One could also think of more invasive changes to the
> block layer (e.g. adding an optional "dispatcher" layer to tightly
> control command ordering ?). And probably a lot more options, But I am
> not yet sure what an appropriate replacement to deadline would be.
> 
> Eventually, the removal of the legacy I/O path may also be the trigger
> to introduce some deeper design changes to blk-mq to accommodate more
> easily zoned block devices or other non-standard block devices (open
> channel SSDs for instance).
> 
> As you can see from the above, working with these drives all day long
> does not make for a clear strategy. Inputs from other here are more than
> welcome. I would be happy to write up all the ideas I have to start a
> discussion so that we can come to a consensus and have a plan.

Doesn't hurt to establish a future plan(s) but we need to deal with the
reality of what we have.  And all we have for this particular issue is
"deadline".  Setting anything else is a bug.

Short of the block layer reinstating the ability for a driver to specify
an elevator: should the zoned driver put a check in place that errors
out if anything other than deadline is configured?

That'd at least save users from a very cutthroat learning curve.

> >>> I don't dispute there is an obvious void for how to properly setup zoned
> >>> devices, but this script is _not_ what should fill that void.
> >>
> >> Good to know! Again, consider it as an alternative to the script.
> >>
> >> I'm happy to adapt the language and supply it only as an example script
> >> developers can use, but we can't leave users hanging as well. Let's at
> >> least come up with a plan which we seem to agree on and document that.
> > 
> > Best to try to get Damien and others more invested in zoned devices to
> > help you take up your cause.  I think it is worthwhile to develop a
> > strategy.  But it needs to be done in terms of the norms of the existing
> > infrastructure we all make use of today.  So first step is making
> > existing tools zoned device aware (even if to reject such devices).
> 
> Rest assured that I am fully invested in improving the existing
> infrastructure for zoned block devices. As mentioned above, applications
> based use of zoned block devices still prevails today. So I do tend to
> work more on that side of things (libzbc, tcmu, sysutils for instance)
> rather than on a better integration with more advanced tools (such as
> LVM) relying on kernel features. I am however seeing rising interest in
> file systems and also in dm-zoned. So definitely it is time to step up
> work in that area to further simplify using these drives.
> 
> Thank you for the feedback.

Thanks for your insight.  Sounds like you're ontop of it.

Mike